got a lot of great answers to "how is the shell involved in handling Ctrl+C" to the effect of one of:

- "the shell is responsible for sending a SIGINT"
- "the shell takes input from the terminal emulator and relays it to the program"
- "it must be handled by the layer above the program, which is the shell"

and while none of those is true, they feel like they totally _could_ be true in a different world. It feels hard to explain why it might be useful to have a more "accurate" mental model

@b0rk

Maybe invert the explanation, start with the terminal driver, cooked vs raw, etc?

@rk yeah I mean it's totally possible to explain all that stuff but... why should someone care? how does it help you get stuff done? it feels like a much more complicated mental model than "terminal emulator -> shell -> process" and it's not 100% obvious to me what value provides (other than being "correct" haha)

it feels like there's this mental model a lot of people have for the terminal which is a little like this (with some more stuff about signals etc)

and it's not "true" that the shell is in between the terminal emulator and programs in this way but it feels more compelling to me in some ways than the reality.

@b0rk What is the correct version? Is the shell at the same level as the (other) programs? Or is the shell a tiny bit special here?

@bartavi it's something like this but this diagram doesn't do a great job of describing the shell's role on its own (or what the terminal driver does)

@b0rk @bartavi
I see now that I was a fool; I had forgotten the golden rule: 'everything is a file'.

@malin @bartavi the files in the terminal context are weird and it involves a lot of `ioctl`s and internal state inside the OS sadly

@b0rk Does it make a nice hierarchy, like the ISO 8-layer model?? That one isn't actually true, but it is a good engineering approximation of the arpanet 3-layer model*.

One layer is the window manager, then the terminal app, then the shell, and then the application.

[* M.A. Padlipsky, The Elements of Networking Style, Chapter 5, A Perspective on the ARPANET Reference Mode, https://archive.org/details/elementsofnetwor00padl]

@davecb yeah I think that's a good analogy

it makes explaining how the terminal works a little weird — I don't want to teach a “wrong” mental model just because it feels simpler (that seems silly), but also the "correct” mental model feels kind of overly complicated and not worth explaining because there's not obviously a lot of real-world value to learning it

not really looking for advice, just thinking out loud

(3/?)

@b0rk I think this is a useful simplification for basic understanding of how most people work in a terminal. The shell is an agent that we use to work with the operating system. You could put another box around the shell and programs for the operating system.

@EricFielding haha the problem is that it's really not true at all, like programs' interactions with the operating system do not go through the shell in any way

@b0rk it would be kind of nice if ctrl+c and friends were handled by the shell, to be fair! After all, the shell handles a lot of job control UI for the user, it would be 'natural' for it to handle the 'hotkeys' too...

@kitten_tech yeah it makes a lot of sense!! And I think you could even implement a shell that works that way

@b0rk @EricFielding on a micro-kernel unixlike many years ago I had to debug and fix an issue with ^C killing the serial driver.

That was quite a lesson in what's going on with terminal drivers and everything around them

In that space the shell is Just Another Program

@b0rk @rk if you replace "shell" in that model with "operating system stuff" (not getting into the details of pseudo-tty and all), the model is good enough, and the naming makes it correct, I think?

@pulkomandy @rk it is true but then it raises a lot of confusing questions about what the role of the shell is exactly and how that all works (and honestly I find the answers to those questions very hard to summarize in a clear way, like “well the terminal driver has a foreground process group, and shell is in charge of managing what the foreground process is”)

feels bad to spend so much talking about what a “foreground process group” is because it's sort of an obscure concept generally

@b0rk This is reminding me that one of my college computer science courses included an assignment to implement features of a basic shell, but that's been a really long time now

@mulad something I've been learning from these replies is that the “implement a shell" project actually in a way causes more confusion because you can implement a shell in a lot of different ways and not all of those are actually the same as how the shells we use work

(which is fine, like it's cool to do things in different ways, but for me implementing a toy shell definitely gave me some slightly wrong ideas about how shells work)

@b0rk@social.jvns.ca I mean... Is it wrong? Shell launches these programs and forwards i/o, so in a way, this is the only technically accurate visualisation (because the process tree actually does look like this)

@b0rk for me terminal has also ssh and tmux, so complicating everything. Also the desktop manager will intercept some keys. A spaghetti flow. I wonder if someone really understands all details (compressing x kbd settings, DM/DE, serial protocol (still not completely hidden), terminal (strange settings), vim and emacs (pushing things at extreme), tmux, …

@cate yeah ssh and tmux make everything SO much more confusing

@addison it is wrong, the shell does not forward i/o, when it forks the process just inherits the same file descriptors and talks to the OS directly :)

(but it doesn't necessarily "matter" to be wrong in that way, like you can get along fine if you think the shell forwards I/o)

@b0rk @rk I guess this is why abstractions are useful? Most programmers never need to learn about microarch internals, etc. Maybe the important part is to know is that "we have a magic box" instead of "this specific but factually incorrect wiring" between components we currently care about?

@buherator totally, the reason i'm interested in talking about the terminal's internals is that I think the abstractions are extremely leaky, like a lot of the basic things ("Ctrl+C”, "editing text") behave very differently depending on the situation and it's hard to understand why without looking under the surface a bit

@b0rk That sounds like a *very* good reason to get into the nitty-gritty details to me!

@b0rk@social.jvns.ca wait oh no

It doesn't just pass the stdin/stdout on fork/execve? I know there's some wonkiness when you're working with ttys but I wasn't really thinking about this

@addison i mean it does kind of make a copy of its stdin/stdout on fork (in the sense that that happens automatically when you fork a process) but like it's not actively involved in forwarding anything beyond just running `fork` and `execve` one time when starting the process

@b0rk

The real issue with "true" vs "clear" is that the shell owns the process the terminal is talking to, but isn't a 100% middleman.

I'd run with clarity over truth, with a footnote that reality is more complex. Ie. Just what you're doing. There is nothing wrong with a mental model that simplifies learning, so long as we all understand there's ugly details under the hood somewhere.

@b0rk@social.jvns.ca Hm. I guess I always think of that as forwarding, but I see how this is a model that doesn't reflect reality... Wonky. I guarantee that this model has weird inconsistencies with reality with seekable fds or debugging multithreading, but for shells, I think this model at least captures the "vibe" of how shells interact with waiting programs (unless they're backgrounded... oh well)

@b0rk I feel like the interplay between TTYs and signals, both of which are hard to reason about (but especially the former) makes this an especially difficult topic, at least for me

@b0rk just like preschool, I want teachers that tell me the truth. I realize the map is never the terrain, so any lesson contains falsehoods, but I think there is always value in being as close to reality as possible.

This isn't advice. It's just what I want as a paying customer. :)

@heathborders yea telling the truth is the only option, I just need to figure out how to do it

@b0rk I appreciate all your content. Thank you for the great work you're doing. Just one feedback If I may: repeating "not looking for advice" or "not looking for answers" in almost every post is sometimes annoying to the reader (sorry I can' find another word to describe it). Can you explain why do you feel the need to repeat this every time?

@imjam mastodon used to be almost unusable for me because I would get a lot of super repetitive replies to all my posts. Adding the disclaimers makes it possible for me to continue using mastodon, the alternative is probably that I would leave

@b0rk this means this disclaimer helped you, fair enough. Happy you stayed, just ignore my feedback then! Have a great day.

after sleeping on it, maybe one “practical” way that this diagram is untrue is that if the shell WERE in the middle of the terminal emulator and the program, I’d expect "canonical mode” (where the program gets the input you type 1 line at a time) to sometimes have more features, like the shell could make arrow keys work instead of printing ^[[D

but in reality programs talk to the OS directly and the OS can't really add new features to canonical mode for compatibility reasons I guess

(4/?)

@b0rk are you familiar with the term “notional machines” the way CS education researchers (and CS educators I think) use it? seems applicable to what you’re talking about here wrt mental models and to what extent it matters if pedagogical explanations are actually conformant to implementations

@chrisamaphone I'm not, thank you! will look it up

@b0rk I guess one thing that does make sense when you know that programs talk directly to the OS is how stdout, stderr, stdin are like files that are accessible system-wide (I think). I mean, the shell doesn't "own" stdout, it just helps you use it

@leifur yeah! i think it's cool that you can run `tty` and then just echo stuff to the file that prints out

@b0rk Given that this diagram isn't true, I'm now realizing that I have no idea how the jobs stuff in bash is implemented.

@ptd yeah it's super weird, if you're interested I think https://github.com/tokenrove/build-your-own-shell/blob/58bba96caa79a7f261913f701e4aadd437fbfe46/stage_3.md#terminal-foreground-process-group has kind of a nice explanation (in the context of writing your own shell)

@b0rk a simpler explanation: programmers are lazy. Why would I want to write a program that spends most of its time copying bits back and forth, being a bottleneck. Those are valuable CPU cycles. Another take: cat file|grep|more|less|gzip|gunzip|wc . How are bits passed between the programs? It's not the shell; that would suck. DOS did not implement pipes per se, but wrote intermediate files, running one program at a time. I.e. the equivalent of cat file >1 ; grep <1 >2 ; more <2 >3 ; etc

@trouble yea totally, I think in this imaginary world the shell would only be involved in interactive input, not pipes (the same way the terminal driver isn't involved with pipes)

@0x2ba22e11 it's sort of amazing that the ecosystem works at all

@b0rk there's various feature requests for fish that involve this model eg https://github.com/fish-shell/fish-shell/issues/6404

@zanchey interesting thanks!

@b0rk
Wouldn't this also impose limitations on programs "running through the shell"? For instance, ctrl-c might be a vim shortcut for you, that does something other than terminate the program. If it were first interpreted by the shell, that would be impossible (or would entail the shell "knowing" to pass the control sequence down in certain circumstances)

@danielittlewood i think in this imaginary world programs would still be able to specify whether the shell should handle signals/input or just pass them through as raw bytes, the same way they do today (vim already tells the OS to just give it the raw bytes from the terminal emulator)