

What if it is and it became unrecoverable ages ago?
What if it is and it became unrecoverable ages ago?
From top-to-bottom:
Skynet from The Terminator
Joshua from WarGames
They are, however, able to inaccurately summarize it in GLaDOS’s voice, which is a strong point in their favor.
to ensure that the technology is “safe, secure and trustworthy.”
None of the really iconic AIs are safe, secure or trustworthy.
If Justin Trudeau gets active on this, you can probably get it to Gulf of Canada (Gulf of Mexico) (Gulf of America).
Wouldnt the sync option also confirm that every write also arrived on the disk?
If you’re mounting with the NFS sync option, that’ll avoid the “wait until close and probably reorder writes at the NFS layer” issue I mentioned, so that’d address one of the two issues, and the one that’s specific to NFS.
That’ll force each write to go, in order, to the NFS server, which I’d expect would avoid problems with the network connection being lost while flushing deferred writes. I don’t think that it actually forces it to nonvolatile storage on the server at that time, so if the server loses power, that could still be an issue, but that’s the same problem one would get when running with a local filesystem image with the “less-safe” options for qemu and the client machine loses power.
NFS doesn’t do snapshotting, which is what I assumed that you meant and I’d guess ShortN0te also assumed.
If you’re talking about qcow2 snapshots, that happens at the qcow2 level. NFS doesn’t have any idea that qemu is doing a snapshot operation.
On a related note: if you are invoking a VM using a filesystem images stored on an NFS mount, I would be careful, unless you are absolutely certain that this is safe for the version of NFS and the specific caching options for both NFS and qemu that you are using.
I’ve tried to take a quick look. There’s a large stack involved, and I’m only looking at it quickly.
To avoid data loss via power loss, filesystems – and thus the filesystem images backing VMs using filesystems – require write ordering to be maintained. That is, they need to have the ability to do a write and have it go to actual, nonvolatile storage prior to any subsequent writes.
At a hard disk protocol level, like for SCSI, there are BARRIER operations. These don’t force something to disk immediately, but they do guarantee that all writes prior to the BARRIER are on nonvolatile storage prior to writes subsequent to it.
I don’t believe that Linux has any userspace way for an process to request a write barrier. There is not an fwritebarrier()
call. This means that the only way to impose write ordering is to call fsync()/sync() or use similar-such operations. These force data to nonvolatile storage, and do not return until it is there. The downside is that this is slow. Programs that are frequently doing such synchronizations cannot issue writes very quickly, and are very sensitive to latency to their nonvolatile storage.
From the qemu(1)
man page:
By default, the cache.writeback=on mode is used. It will report data writes as completed as soon as the data is present in the host page cache. This is safe as long as your guest OS makes sure to correctly flush disk caches where needed. If your guest OS does not handle volatile disk write caches correctly and your host crashes or loses power, then the guest may experience data corruption. For such guests, you should consider using cache.writeback=off. This means that the host page cache will be used to read and write data, but write notification will be sent to the guest only after QEMU has made sure to flush each write to the disk. Be aware that this has a major impact on performance.
I’m fairly sure that this is a rather larger red flag than it might appear, if one simply assumes that Linux must be doing things “correctly”.
Linux doesn’t guarantee that a write to position A goes to disk prior to a write to position B. That means that if your machine crashes or loses power, with the default settings, even for drive images sorted on a filesystem on a local host, with default you can potentially corrupt a filesystem image.
https://docs.kernel.org/block/blk-mq.html
Note
Neither the block layer nor the device protocols guarantee the order of completion of requests. This must be handled by higher layers, like the filesystem.
POSIX does not guarantee that write() operations to different locations in a file are ordered.
https://stackoverflow.com/questions/7463925/guarantees-of-order-of-the-operations-on-file
So by default – which is what you might be doing, wittingly or unwittingly – if you’re using a disk image on a filesystem, qemu
simply doesn’t care about write ordering to nonvolatile storage. It does writes. it does not care about the order in which they hit the disk. It is not calling fsync()
or using analogous functionality (like O_DIRECT
).
NFS entering the picture complicates this further.
https://www.man7.org/linux/man-pages/man5/nfs.5.html
The sync mount option The NFS client treats the sync mount option differently than some other file systems (refer to mount(8) for a description of the generic sync and async mount options). If neither sync nor async is specified (or if the async option is specified), the NFS client delays sending application writes to the server until any of these events occur:
Memory pressure forces reclamation of system memory resources. An application flushes file data explicitly with sync(2), msync(2), or fsync(3). An application closes a file with close(2). The file is locked/unlocked via fcntl(2). In other words, under normal circumstances, data written by an application may not immediately appear on the server that hosts the file. If the sync option is specified on a mount point, any system call that writes data to files on that mount point causes that data to be flushed to the server before the system call returns control to user space. This provides greater data cache coherence among clients, but at a significant performance cost. Applications can use the O_SYNC open flag to force application writes to individual files to go to the server immediately without the use of the sync mount option.
So, strictly-speaking, this doesn’t make any guarantees about what NFS does. It says that it’s fine for the NFS client to send nothing to the server at all on write(). The only time a write() to a file makes it to the server, if you’re using the default NFS mount options. If it’s not going to the server, it definitely cannot be flushed to nonvolatile storage.
Now, I don’t know this for a fact – would have to go digging around in the NFS client you’re using. But it would be compatible with the guarantees listed, and I’d guess that probably, the NFS client isn’t keeping a log of all the write()s and then replaying them in order. If it did so, for it to meaningfully affect what’s on nonvolatile storage, the NFS server would have to fsync() the file after each write being flushed to nonvolatile storage. Instead, it’s probably just keeping a list of dirty data in the file, and then flushing it to the NFS server at close().
That is, say you have a program that opens a file filled with all ‘0’ characters, and does:
At close() time, the NFS client probably doesn’t flush “1” to position 1, then “1” to position 5000, then “2” to position 1, then “2” to position 5000. It’s probably just flushing “2” to position 1, and then “2” to position 5000, because when you close the file, that’s what’s in the list of dirty data in the file.
The thing is that unless the NFS client retains a log of all those write operations, there’s no way to send the writes to the server in a way that avoid putting the file into a corrupt state if power is lost. It doesn’t matter whether it writes the “2” at position 1 or the “2” at position 5000. In either case, it’s creating a situation where, for a moment, one of those two positions has a “0”, and the other has a “2”. If there’s a failure at that point – the server loses power, the network connection is severed – that’s the state in which the file winds up in. That’s a state that is inconsistent, should never have arisen. And if the file is a filesystem image, then the filesystem might be corrupt.
So I’d guess that at both of those two points in the stack – the NFS client writing data to the server, and the server block device scheduler, permit inconsistent state if there’s no fsync()/sync()/etc being issued, which appears to be the default behavior for qemu
. And running on NFS probably creates a larger window for a failure to induce corruption.
It’s possible that using qemu’s iSCSI backend avoids this issue, assuming that the iSCSI target avoids reordering. That’d avoid qemu going through the NFS layer.
I’m not going to dig further into this at the moment. I might be incorrect. But I felt that I should at least mention it, since filesystem images on NFS sounded a bit worrying.
Honestly, I’m a little surprised that a smartphone user wouldn’t have familiarity with the concept of files, setting aside the whole familiarity-with-a-PC thing. Like, I’ve always had a file manager on my Android smartphone. I mean, ok…most software packages don’t require having one browse the file structure on the thing. And many are isolated, don’t have permission to touch shared files. Probably a good thing to sandbox apps, helps reduce the impact of malware.
But…I mean, even sandboxed apps can provide file access to the application-private directory on Android. I guess they just mostly don’t, if the idea is that they should only be looking at files in application-private storage on-device, or if they’re just the front end to a cloud service.
Hmm. I mean, I have GNU/Linux software running in Termux, do stuff like scp
from there. A file manager. Open local video files in mpv
or in PDF viewers and such. I’ve a Markdown editor that permits browsing the filesystem. Ditto for an org-mode editor. I’ve a music player that can browse the filesystem. I’ve got a directory hierarchy that I’ve created, though simpler and I don’t touch it as much as on the PC.
But, I suppose that maybe most apps just don’t expose it in their UI. I could see a typical Android user just never using any of the above software. Not having a local PDF viewer or video player seems odd, but I guess someone could just rely wholly on streaming services for video and always open PDFs off the network. I’m not sure that the official YouTube app lets one actually save video files for offline viewing, come to think of it.
I remember being absolutely shocked when trying to view a locally-stored HTML file once that Android-based web browsers apparently didn’t permit opening local HTML files, that one had to set up a local webserver (though that may have something to do with the fact that I believe that by default, with Web browser security models, a webpage loaded via the file://
URI scheme has general access to your local filesystem but one talking to a webserver on localhost does not…maybe that was the rationale).
There was something of an earlier effort to do MOOCs. My impression is that they didn’t take off, because I stopped hearing about them. But I don’t really follow current education, so…
I think that at some point, dramatic improvements to education are probably doable, and that we probably have the technology today.
But I’m kind of skeptical that AI is really the missing piece, at least given the state that it’s in today.
PNG is really designed for images that are either flat color or use an ordered dither. I mean, we do use it for photographs because it’s everywhere and lossless, but it was never really intended to compress photographs well.
There are formats that do aim for that, like lossless JPEG and one of the WebP variants.
TIFF also has some utility in that it’s got some sort of hierarchical variant that’s useful for efficiently dealing with extremely-large images, where software that deals with most other formats really falls over.
But none of those are as universally-available.
Also, I suppose that if you have a PNG image, you know that – well, absent something like color reduction – it was losslessly-compressed, whereas all of the above have lossless and lossy variants.
Oh, yeah, not saying that they were the first filesystems, just that I can remember that transition on the personal computer.
I would guess that at least part of the issue there is also that the data isn’t all that useful unless it’s also exported to some format that other software can read. That format may not capture everything that the native format stores.
In another comment in this thread, I was reading the WP article on Adobe Creative Cloud, which commented on the fact that the format is proprietary. I can set up some “data storage service”, and maybe Adobe lets users export their Creative Cloud data there. Maybe users even have local storage.
But…then, what do you do with the data? Suppose I just get a copy of the native format. If nothing other than the software on Adobe’s servers can use it, that doesn’t help me at all. Maybe you can export the data, export to an open format like a PNG or something, but you probably don’t retain everything. Like, I can maybe get my final image out, but I don’t get all the project workflow stuff associated with the work I’ve done. Macros, brushes, stuff broken up into layers, undo history…
I mean, you have to have the ability to use the software to maintain full use of the data, and Adobe’s not going to give you that.
I think the first filesystems had flat layout (no directories), but also had different file types for a library, an executable, a plaintext file. Then there were filesystems where directories could only list files, not other directories.
The original Macintosh filesystem was flat, and according to WP, used for about two years around the mid-1980s. I don’t think I’ve ever used it, personally.
https://en.wikipedia.org/wiki/Macintosh_File_System
MFS is called a flat file system because it does not support a hierarchy of directories.
They switched to a new, hierarchical filesystem, HFS, pretty soon.
I thought that Apple ProDOS’s file system – late 1970s to early 1980s – was also flat, from memory. It looks like it was at one point, though they added hierarchical support to it later:
https://en.wikipedia.org/wiki/Apple_ProDOS
ProDOS adds a standard method of accessing ROM-based drivers on expansion cards for disk devices, expands the maximum volume size from about 400 kilobytes to 32 megabytes, introduces support for hierarchical subdirectories (a vital feature for organizing a hard disk’s storage space), and supports RAM disks on machines with 128 KB or more of memory.
Looks like FAT, used by MS-DOS, early 1980s, also started out flat-file, then added hierarchical support:
https://en.wikipedia.org/wiki/File_Allocation_Table
The BIOS Parameter Block (BPB) was introduced with PC DOS 2.0 as well, and this version also added read-only, archive, volume label, and directory attribute bits for hierarchical sub-directories.[24]
Yes. Newest version of mint uses pipewire as audio engine, and if you combine that with the good old pulsaudio controller (which works with pipewire via a plugin), you can micro which app input/output goes to which device.
I believe that you can also fiddle with it at the pipewire level, though I’ve not played with that.
pokes around
Yeah, qpwgraph
is what I’m thinking of.
That’s got kind of a JACK-style flowchart patchbay interface, though. I’d guess that most people want what the PulseAudio pavucontrol
provides in terms of UI.
EDIT: Ah, someone else mentioned it below already.
The average person does not deal with files anymore. Many people use online applications for everything from multimedia to documents, which happily abstract away the experience of managing file formats.
I remember someone saying that and me having a hard time believing it, but I’ve seen several people say that.
https://www.theverge.com/22684730/students-file-folder-directory-structure-education-gen-z
Catherine Garland, an astrophysicist, started seeing the problem in 2017. She was teaching an engineering course, and her students were using simulation software to model turbines for jet engines. She’d laid out the assignment clearly, but student after student was calling her over for help. They were all getting the same error message: The program couldn’t find their files.
Garland thought it would be an easy fix. She asked each student where they’d saved their project. Could they be on the desktop? Perhaps in the shared drive? But over and over, she was met with confusion. “What are you talking about?” multiple students inquired. Not only did they not know where their files were saved — they didn’t understand the question.
Gradually, Garland came to the same realization that many of her fellow educators have reached in the past four years: the concept of file folders and directories, essential to previous generations’ understanding of computers, is gibberish to many modern students.
https://old.reddit.com/r/AskAcademia/comments/1dkeiwz/is_genz_really_this_bad_with_computers/
The OS interfaces have followed this trend, by developing OS that are more similar to a smartphone design (Windows 8 was the first great example of this). And everything became more user-friendly (my 65+ yo parents barely know how to turn on a computer, but now, use apps for the bank and send emails from their phone). The combined result is that the younger generations have never learned the basic of how a computer works (file structure, file installation…) and are not very comfortable with the PC setup (how they prefer to keep their notes on the phone makes me confused).
So the “kids” do not need to know these things for their daily enjoyment life (play videogames, watch videos, messaging… all stuff that required some basic computer skills even just 10 years ago, but now can be done much more easily, I still remember having to install some bulky pc game with 3 discs) and we nobody is teaching them because the people in charge thought “well the kids know this computer stuff better than us” so no more courses in elementary school on how to install ms word.
For a while I was convinced my students were screwing with me but no, many of them actually do not know the keyboard short cuts for copy and paste. If it’s not tablet/phone centric, they’re probably not familiar with it.
Also, most have used GSuite through school and were restricted from adding anything to their Chrome Books. They’ve used integrated sites, not applications that need downloading. They’re also adept at Web 3.0, creation stuff, more than professional type programs.
As much as boomers don’t know how to use PCs because they were too new for them, GenZs and later are not particularly computer savvy because computers are too old for them.
I can understand some arguments that there’s always room to advance UI paradigms, but I have to say that I don’t think that cloud-based smartphone UIs are the endgame. If one is going to consume content, okay, fine. Like, as a TV replacement or something, sure. But there’s a huge range of software – including most of what I’d use for “serious” tasks – out there that doesn’t fall into that class, and really doesn’t follow that model. Statistics software? Software development? CAD? I guess Microsoft 365 – which I have not used – probably has some kind of cloud-based spreadsheet stuff. I haven’t used Adobe Creative Cloud, but I assume that it must have some kind of functionality analogous to Photoshop.
kagis
Looks like off-line Photoshop is dead these days, and Adobe shifted to a pure SaaS model:
https://en.wikipedia.org/wiki/Adobe_Creative_Cloud#Criticism
Shifting to a software as a service model, Adobe announced more frequent feature updates to its products and the eschewing of their traditional release cycles.[26] Customers must pay a monthly subscription fee. Consequently, if subscribers cancel or stop paying, they will lose access to the software as well as the ability to open work saved in proprietary file formats.[27]
shakes head
Man.
And for that matter, I’d think that a lot of countries might have concerns about dependence on a cloud service. I mean, I would if we were talking about China. I’m not even talking about data security or anything – what happens if Country A sanctions Country B and all of Country B’s users have their data abruptly inaccessible?
I get that Internet connectivity is more-widespread now. But, while I’m handicapped without an Internet connection, because I don’t have access to useful online resources, I can still basically do all of the tasks I want to do locally. Having my software unavailable because the backend is unreachable seems really problematic.
Do you use a macro keyboard for shortcuts?
No. I think that macro functionality is useful, but I don’t do it via the physical keyboard.
My general take is that chording (pressing some combination of keys simultaneously) that lets one keep one hands on the home row is faster than pressing one key. So, like, instead of having separate capital and lowercase letter keys, it’s preferable to have “shift” and just one key.
I think that the main arguments for dedicated keys that one lifts one hands for would be for important but relatively-infrequently-used keys that people don’t use enough to remember chorded combinations for – you can just throw the label on the button as a quick reference. Like, we don’t usually have Windows-Alt-7 on a keyboard power on a laptop, but instead have a dedicated power button.
Maybe there’s a use to have keyboard-level-programmed macros with chording, as some keyboards can do…but to me, the use case seems pretty niche. If you’re using multiple software environments (e.g. BIOS, Windows, Linux terminal, whatever) and want the same functionality in all of them (e.g. a way to type your name), that might make some sense. Or maybe if you’re permitted to take a keyboard with you, but are required to use a computer that you can’t configure at the software level, that’d provide configurability at a level that you have control over.
In general, though, I’m happier with configuring stuff like that on the computer’s software; I don’t hit those two use cases, myself.
considers tradeoffs
So, if it were me, I’d probably use a text editor, as the big text editors are going to be able to do the rest of those. I’ve authored text in a number of lightly-annotated-text formats like Markdown and similar (AsciiDoc, Docbook, Markdown), and I always did it in emacs
, and generally vim
has analogous functionality.
If you don’t mind me asking, what is the use case for this? Markdown originally had the design goal to be easily-editable as plain text – that is, one could view it as plain text almost as well as in a rendered form. That was kind of its selling point relative to many other rich text markup languages; it was intended to let a user edit and read it in a plain-text editing environment, without even colorization and such.
Markdown, unlike, say, PDF or Microsoft Word or something, is intended to be display-device-agnostic. That is, if you distribute a Markdown file to others, what the end user will see may differ from what you see, because Markdown intentionally abstracts the specifics of how the material is displayed. Normally, WYSIWYG is mostly-useful for formats that don’t do this. If you’re using Markdown to author PDFs or printed pages or something and then using that format for distribution, I get that. But if you’re handing out Markdown, what you see might not be what the end user sees; screen size differences, typeface differences, other things may pretty-dramatically change what they see. Even above-and-beyond device differences, the Lemmy Web UI, Mbin, the Lemmy clients that I’ve used, Reddit, and GitHub all use intentional variations on the basic Markdown format. IIRC from last time I used pandoc
, it supports multiple of the different dialects, but even it can’t provide a representation for every one. Lemmy recently ripped out the auto-renumbering of numbered lists. IIRC either Reddit or Lemmy disabled huge top-level headers after people abused them to flood threads (though maybe that’s a per-community/subreddit CSS thing). Reddit doesn’t support Markdown’s syntax for inline images (well, old.reddit doesn’t, at any rate, and I haven’t tested new.reddit). Many websites and client software packages that present Markdown permit a user to view it in light-on-dark or dark-on-light or have other theming options.
Is it to make sure that there are no errors in the Markdown leading to some kind of wonky display, say (like a table row missing a trailing pipe or something)? There may be a non-editable-WYSIWYG way to accomplish that that might work with text editors. pandoc
may have the ability to emit errors, and it looks like there are Markdown linting packages. It may be possible to rig those up to a text editor to highlight any errors. If I were doing it in emacs
, I’d guess that flycheck
can run a command in the background during idle time to check for errors and flag them. I don’t know how vim
does it, but I’m sure that it has an analogous feature.
kagis
It looks like flycheck already does support running markdownlint in the background to highlight errors in Markdown, actually:
https://www.flycheck.org/en/latest/languages.html
Supported Languages
Markdown
markdown-markdownlint-cli
Check Markdown with markdownlint-cli.
goes to try it out
On my system, I just needed to install markdownlint, and then enable the flycheck minor mode (M-x flycheck-mode
) when in markdown-mode. Then emacs starts highlighting errors in the Markdown text, and one can get a list of errors in the document and jump to each.
It looks like markdownlint is pretty nitpicky out of the box (warns about line length, trailing whitespace, multiple consecutive blank lines), but I expect that that’s configurable. It also did pick up on table column count mismatches, which is the main thing that I can think of to ask it for.
I also see a couple of vim extensions for markdownlint. Can’t speak as to their functionality, but there’s some level of integration there as well.
The side-by-side view doesn’t do it for me, I’d more likely than not have multiple windows open with different documents instead.
That’ll probably rule out text editors like emacs if you don’t want side-by-side. Emacs has some functionality that can do some styling, but you probably won’t have a purely WYSIWYG mode for, say, tables. It looks like emacs has some way to translate org-mode tables to Markdown, but that’s probably not quite what you want.
It should do autocomplete, syntax highlighting, bracket closing, live spell checking in a variety of languages, launch quickly, be rock solid when faced with a massive log file and allow me to add menu-items to run bash scripts that do things like calculate the time it would take me to read out the text at my normal podcast reading voice or covert weird characters into hrml-entities.
That’ll rule out most “small” programs targeting specifically Markdown.
Depends on what you mean by “massive” log files. If you mean you require out-of-memory editing – the ability to load only a small portion of the document into memory, which is probably going to be necessary once you exceed your machine’s main memory – then you’re looking at a small set of software. Some hex editors, emacs can use vlf
(which will constrain other features available), a few programs targeting specifically this feature.
I haven’t looked at heavyweight word processors, but some may have reasonable support for at least many of those, stuff like LibreOffice. They probably won’t open quickly, but there are a few programs capable of speeding up startup by leaving a daemon running, just opening something in that daemon, like emacs
, urxvt
, etc. You can possibly do that or just leave a blank document open on another workspace.
Emacs’s Markdown mode has two options for preview:
C-c C-c p
(Control-C Control-C p) runs markdown-preview
, which will open a preview in a new window
C-c C-c l
runs markdown-live-preview mode
, which will show an updated-as you edit preview next to the text.
In addition to built-in functionality, in my emacs setup, I also personally bind C-c a k
to run Make. In my init.el
:
(global-set-key (kbd "C-c a k") 'compile)
That way, if you have any sort of project – which could hypothetically be a Markdown file – and a Makefile for it in the same directory, it’ll build it. An example Makefile:
all: foo.pdf
%.pdf: %.md
pandoc -f markdown -t pdf $< -o $@
Editing foo.md
in emacs and hitting C-c a k
will regenerate the pdf using pandoc
with that setup. It sounds like you’re familiar with pandoc
.
If you have evince
running on foo.pdf, it’ll monitor changes to the displayed pdf file, and then just update its display if the file changes.
Don’t they support video posts?
kagis
Hmm. Apparently so.
https://help.tumblr.com/knowledge-base/posting-video/
Well, it’s not an open platform, but I guess it may be a platform with the resources to serve some serious video, and it’ll be on the Fediverse.
We have been having a lot of discussions about what it would take to get video on the Fediverse at more than PeerTube scale.
I don’t use tumblr, don’t know if they provide a video-centric interface, but I imagine that one could always write a software package to index those videos and link to them. Maybe PeerTube can already do that, haven’t played with it enough to know.