Wow, POSIX specifies that the type of the nanoseconds field in struct timespec must be long. Did they have a competition to pick the worst possible choice for that type? It's a value that must be able to hold a (maximum of 30-bit) value from 0 to 999,999,999. So they picked a signed type that is at least 32 bits, but sometimes larger.

@david_chisnall LOL i was actually just looking at timespec right this minute and had questions about the type choices

@dysfun It appears that we have made similar poor life choices.

@david_chisnall unless you're doing green threads too, i feel like you might be getting off better than me today

@dysfun Implementing a wall-clock subsystem with pluggable clock sources for CHERIoT RTOS so... maybe. Last time I implemented green threads, it was more fun than this.

@david_chisnall I wonder if they standardised it from implementations that decided to use -1 for "nanoseconds unsupported" or something like that.

@andyprice

It's curious because it's a newer addition. The older timeval provided microseconds and it made both fields a platform-defined typedef, so you can pick whatever makes sense for you.

@david_chisnall It would be useful to represent leap seconds and thus solve problems elsewhere in the API where days are assumed to be exactly 86,400 seconds, but then I see that it specifically has restricted range and thus snatches defeat out of the jaws of victory.

This is a double-clawed PHP hammer with "POSIX" crayoned onto it.

@david_chisnall @dysfun

Steal FreeBSD's timecounters.

@bsdphk @dysfun

They're much more complexity than a tiny embedded device needs, but they've provided some inspiration.

@david_chisnall i'm not having fun. also i'm not supporting anything but x86-64 because i have to do this for every fucking platform and i don't have any users yet. and i am of course doing all of this because we can't have nice things in c.

but no, i'm attempting to do optional per-gt cpu tracking. to go with the stack low watermark (taken at pause time) to help out with the fact i don't really know shit about stack size because there's a compiler in the way. because we can't have nice things in c. etc.

@dysfun One of my current time sinks is trying to prevent fibres getting into C++29. It's very easy to implement them, as long as you don't care how much existing C/C++ you subtly break. I'd much rather people don't use green threads for C/C++ unless they really know what they're doing.

If you're happy with the limitations (e.g. don't use any existing standard locks because they introduce self-deadlock possibilities, don't rely on _Thread_local / thread_local being lexically scoped, don't expect state associated with a thread such as the signal mask to not be modified out from under you), there's a boost library that does a reasonable job.

@david_chisnall There is so much we could do so much better if we took all the lessons from POSIX, Plan9, Windows, whatever and started building from scratch in the modern day.

I completely understand that in modern world that is extremely unlikely to happen, but building on a questionable foundation as much as we do without even realising the foundation might not be what we'd hoped for is kinda incredible.

With the modern hyper-producitity-seeking institutions, I don't think I'll live to see this change, but it's insane to think how much we already know we could improve on the fundamentals if we just tried to.

@ananas That's more or less what we're doing in CHERIoT RTOS, with some compatibility layers for porting existing stuff. It's much more feasible for embedded, so that's also a place you can demonstrate things that you wish bigger systems did.

@david_chisnall @dysfun

Your "embedded device" is 10 times faster than the i386/i486 I developed timecounters on :-)

@bsdphk @dysfun

But almost certainly has less RAM and less diversity of hardware.

@david_chisnall @dysfun

... for now. (?)

@bsdphk @dysfun

Maybe. Compute for embedded has been growing at roughly half the rate of desktop for most of the last 30 years, but memory has been growing a lot more slowly. Which makes it quite fun because you don’t really have to care too much about performance: the CPU is effectively infinitely fast for a lot of workloads that can fit in memory.

@david_chisnall in fact i do not care about how much existing C/C++ i subtly break because it's all new code developed for a particular project.

i'm not writing c++. i can just about handle C's UB rules, but nobody can program C++.

@dysfun @david_chisnall Writing that down: „nobody can program c++“

I think it‘s beautiful. 😌

@icing @david_chisnall i mean i've seen talks by members of the c++ iso committee where they say members cannot agree on what the standard already says, let alone where it should go.

@dysfun @david_chisnall Agree. It‘s totally cursed. But excellent job security for the committee.

@icing @dysfun

From the work we did on the de-facto C standard, I’m pretty sure no one can program C either. One of the things we did was send questions to compiler writers, members of the standards committee, and systems programmers. It was terrifying the number of questions that we set where:

• Standards-committee members said the language doesn’t guarantee the behaviour.
• GCC and Clang developers said that it was impossible to guarantee the behaviour in a vaguely performant implementation.
• Systems programmers said that the behaviour was absolutely critical and everything depended on it, and without it everything would break.

Everyone who uses C++ defines a subset to use. Everyone who uses C defines a superset to use.

@david_chisnall @dysfun Every standard and its implementations meet potholes, not just programming languages.

The complexity of the standard determines if this is somewhat manageable or a desaster.

C89 is what we use in #curl, sprinkled with some stuff from C99. We don‘t go beyond this and probably never will, because the cost/benefit does not justify it.

Our battlefield is platforms and interop with other implementations and dependencies.

@icing @dysfun

With C89, you cannot write multithreaded code without relying on either vendor extensions or undefined behaviour. And maybe that’s fine for libcurl, but it isn’t for most programs that use libcurl.

@david_chisnall @dysfun We use pthreads for DNS resolving. Where would be the problem with that? Curious.

@icing @dysfun

Prior to C11, there is no way of enforcing ordering of memory operations. The volatile keyword means that the compiler may not elide loads or stores to a specific object and may not reorder accesses to that object, but it may reorder any other memory access relative to that access.

This means that you cannot implement any of the pthread locking primitives in ISO C89. And no one did, they used GNU extensions, inline assembly (itself a vendor extension) or some combination of the two.

If you just use pthread locks (or equivalent), the calling code can be C89 if you’re careful. But there is no mechanism in the language’s abstract machine to prevent reordering across lock acquisition, you are relying on the fact that the compiler doesn’t look inside the lock and unlock calls and must assume that they may modify or read any memory location that it cannot prove is unaliased.

@david_chisnall @dysfun I see. Thanks.

@icing @dysfun

C99 also introduced a bunch of important improvements. I’ve recently contributed to a project that used C89 and trying to get some thing working without access to variadic macros was painful. More importantly, it introduced intptr_t and, prior to that, there was no safe way of storing a pointer in an integer (sadly, there remains no non-implementation-defined way of doing arithmetic on the result. I have a draft paper for C++29 that will fix that, as well as introducing a non-provenance-carrying type that can hold a address, which we’ll hopefully back port to C at some point. The paper that makes intptr_t mandatory [but still leaves most semantics implementation defined] was approved for C++29, so will hopefully be in C at some point as well).

Oh, and C99 let you put declarations inline, which lets you have a style guideline that requires variable initialisation to be at the same place as the declaration, which means you can eliminate uninitialised use entirely. Without that, you’re reliant on compiler dataflow analysers that, these days, work most of the time. It also introduced a bunch of Fortran-envy poorly designed floating-point things, which is why it took so long for most implementations to claim compliance.

C11, in addition to antomics, added the ability to set alignment for types and a way of writing generic macros, both of which are nice for providing usable APIs, and static assertions.

My favourite thing in C23 is that it made () as a parameter list equivalent to (void), which C++ did a few decades earlier. This meant that, finally, the ‘I forgot C was stupid and now i have turned off type checking and have a load of footguns for callers’ thing was gone. That, alone, is why i set the default C mode to C23 for anything compiled targeting CHERIoT.

@david_chisnall @dysfun

Interestingly, we just got an issue/PR regarding pointers in CHERI being 128bit wide which throws some assumptions.

We added use of stdint.h quite recently and are clearing up our 'int' types which is good for the code.

Inline declarations are a blessing and a curse. I like them for loops, but excessive use requires auto cleanup/defer.

We generally use a 'goto out;' in functions to the one place where cleanup happens. inline does not work well with that.

@icing @david_chisnall CHERI is going to cause me to rewrite a lot of code in the future if it ever gets to big enough chips to make it worth running things like databases on. it's going to make a number of my atomics tricks impossible and i'll have to move to locking.

while i of course wish CHERI the greatest success, if you could stay out of desktop class processors for a while, David...

@dysfun @icing

What atomic tricks are you doing that wouldn’t work with CHERI?

@david_chisnall @icing i can't fit a pointer in 64 bits, so i can't just swap the pointer into a 64 bit atomic without taking a lock. i could use something like cmpxchg16b if it's available.

@dysfun @icing

Every CHERI target supports _Atomic(void*) and _Atomic(uintptr_t). Changing your code to use uintptr_t instead of unit64_t will make it CHERI-clean and work on the same way on non-CHERI systems.

@david_chisnall @icing what, like cmpxchg16b or like we make it work, but it'll be slow? making it compile under CHERI is probably not a huge deal, but making it fast on CHERI may be another matter.

@dysfun @icing

Yes, register-width atomics. Atomics within a cache line are trivial to implement. Morello had them and all of them and the RISC-V atomics spec has them. If you can do atomics at all, a CHERI system will do capability-width atomics. In particular, all capability loads and stores must be (relaxed-consistency) atomic, so providing the extra read-modify-write operations doesn’t add any noticeable complexity on a system with caches.

At the really-tiny end, CHERIoT provides a lightweight way of exposing functions that run with interrupts disabled and so the atomic helpers for single-core microcontrollers are very cheap.

@david_chisnall

What's the alternative? According to the standard, C is *allowed* but not *required* to provide fixed-width integers, so if you want to pick an integral type that's (1) required to exist by the C standard, (2) definitely able to hold numbers up to 10^9 ~ 2^30, and (3) as small as possible otherwise, it sounds like either long or unsigned long are your options.

I guess you could argue for the latter since tv_nsec isn't supposed to be negative, but given how much of a footgun C's semantics for arithmetic are once you start getting unsigned types into the mix I'm kind of glad they didn't.

@danielmclaury For all of the other things in this part of the spec, the fields are some typedef that the implementation can pick. This means, even if it doesn’t have the standard fixed-width integer types, it can still choose between the built-in types and pick a sensible one, such as unsigned int (on ILP32 and LP64 platforms).

@david_chisnall

I guess strictly speaking `int_least32t` and `int_fast32_t` are options, but of course at least per the standard each of those is also " a signed type that is at least 32 bits, but sometimes larger."

@david_chisnall

Also it looks like POSIX says that they defer to the C standard for this definition, and it looks like the C standard was long prior to C23 but is "an implementation-defined signed integer type that can represent integers in [0, 999999999]" in C23.

@david_chisnall @icing @dysfun Borrowing this quote for training materials I'm working on, I hope you don't mind

@jsbarretto @icing @dysfun

This is the paper from that work: https://dl.acm.org/doi/10.1145/2908080.2908081

@david_chisnall @icing must be relaxed? i assume you mean i have to insert a fence afterwards if i want acqrel ?

@david_chisnall @icing oh wait, you mean non-atomic stores of capabilities don't exist?

@dysfun @icing

Atomic, in C/C++ means two things:

• The operation is indivisible (what most people mean by atomic).
• The operation has some ordering rules.

Most of the atomics spec revolves around the ordering: what are sequences of loads in one thread allowed to see with respect to stores from another thread?

The weakest form of atomic in the C/C++11 model is relaxed. This means another thread reading at the same location is guaranteed to see either the value that was there before the store or the value that was stored (or the value of a later store). It is not required to be ordered with respect to other stores. If you do two relaxed-consistency atomic stores from one thread, another thread is permitted to see any interleaving of them. But it is not allowed to see tearing: each store either happens or does not, there is no way of observing half of the store.

In the C abstract machine, if you store a pointer (or an integer) to a global and read it from another thread, that’s UB (data races are UB) and the reason for this is systems like the 386 SX with 32-bit registers and a 16-bit memory bus. In an SMP 386 SX (I don’t think anyone ever built one, the SMP 386 DXs were weird enough), one processor storing a register-sized value would allow another to observe half of the before value and half of the after value.

But on most modern systems, loads and stores of register-sized things are relaxed-consistency atomic. If you store a value from a register, the cache line is locked in exclusive mode, the store happens, then the cache line is released and the result is visible to other cores. The store queue and cache coherency protocol may make this visible with respect to other cache lines (or partial cache lines) as the architecture’s memory model defines. For Arm there’s a lot of freedom for x86 there’s much less.

CHERI systems follow the same principle. Stores from a capability register are relaxed-consistency atomic: other threads see either the old or new value. If you can do that and you can do atomic read-modify-write operations on other smaller-than-a-cache-line quantities, doing the full set of atomics for capabilities is trivial microarchitecturally.

There’s a proposal in C++ to treat all builtin-type loads and stores as relaxed-consistency atomic, because it turns out that a lot of people assume this in their code and it doesn’t actually improve performance for optimisers to be able to assume it doesn’t. That’s fine for CHERI but it breaks Fil-C.

@david_chisnall they wanted to pick a type that would work for a long time :)

@david_chisnall POSIX has a history of bad decisions regarding timekeeping ranging from abysmal to outright illegal

@david_chisnall @icing @dysfun

that it made () as a parameter list equivalent to (void)

That is one of the things I hate the most in C23, and that make me pretty much refuse it.

They should have made it an error instead of silently changing the behaviour. This is a severe class of inacceptable.

@icing @david_chisnall @dysfun I’m moving away from stdint.h again. Nowhere is guaranteed that the rank of, say, int32_t is not a lower rank of int.

@icing @david_chisnall @dysfun OpenLDAP's libldap/liblber are required to only use C89, to allow them to build on legacy systems.

We finally allowed variadic macros for slapd but otherwise there's not much in C99 that's useful to us.

Inline declaration of loop index is decidedly unhelpful, since we frequently need to know the loop index after it terminates.

@hyc @icing @dysfun

C89, to allow them to build on legacy systems.

I hear this a lot, but always in those vague terms. What are the ‘legacy systems’ that have C89 but not C99 or C11 compilers?

@icing @david_chisnall the great thing about writing brand new code is i can just use c23. i might even be able to just do that until some fuckwit demands windows support (msvc supports most of C11)

@dysfun @icing @david_chisnall You can use clang or gcc on Windows. Maybe you can even cross-compile from Linux. For my purposes, this works nicely.

@uecker @dysfun @icing

Clang has pretty good MS ABI compatibility, including for C++. For Objective-C, we even made exceptions interoperate with SEH and C++ exceptions.