13 ms·

I would strongly implore people not to follow the example this post suggests, and write code that relies on this monotonicity. The reason for this is simple: t

I would strongly implore people not to follow the example this post suggests, and write code that relies on this monotonicity. The reason for this is simple: the documentation doesn't promise this property. Moreover, even if it did, the RFC for UUIDv7 doesn't promise this property. If you decide to depend on it, you're setting yourself up for a bad time when PostgreSQL decides to change their implementation strategy, or you move to a different database. Further, the stated motivations for this, to slightly simplify testing code, are massively under-motivating. Saving a single line of code can hardly be said to be worth it, but even if it were, this is a problem far better solved by simply writing a function that will both generate the objects and sort them. As a profession, I strongly feel we need to do a better job orienting ourselves to the reality that our code has a tendency to live for a long time, and we need to optimize not for "how quickly can I type it", but "what will this code cost over its lifetime".

3eb7988a1663 2y ago
I too am missing the win on this. It is breaking the spec, and does not seem like it offers a significant advantage. In the eventual event where you have a collection of UUID7 you are only ever going to be able to rely on the millisecond precision anyway.
- sbuttgereit 2y ago
  You say it's breaking the spec, but is it? From https://www.rfc-editor.org/rfc/rfc9562.html#name-uuid-version-7 https://www.rfc-editor.org/rfc/rfc9562.html#name-uuid-versio...: "UUIDv7 values are created by allocating a Unix timestamp in milliseconds in the most significant 48 bits and filling the remaining 74 bits, excluding the required version and variant bits, with random bits for each new UUIDv7 generated to provide uniqueness as per Section 6.9. Alternatively, implementations MAY fill the 74 bits, jointly, with a combination of the following subfields, in this order from the most significant bits to the least, to guarantee additional monotonicity within a millisecond: 1. An OPTIONAL sub-millisecond timestamp fraction (12 bits at maximum) as per Section 6.2 (Method 3). 2. An OPTIONAL carefully seeded counter as per Section 6.2 (Method 1 or 2). 3. Random data for each new UUIDv7 generated for any remaining space." Which the referenced "method 3" is: "Replace Leftmost Random Bits with Increased Clock Precision (Method 3): For UUIDv7, which has millisecond timestamp precision, it is possible to use additional clock precision available on the system to substitute for up to 12 random bits immediately following the timestamp. This can provide values that are time ordered with sub-millisecond precision, using however many bits are appropriate in the implementation environment. With this method, the additional time precision bits MUST follow the timestamp as the next available bit in the rand_a field for UUIDv7."
- throw0101c 2y ago
  > It is breaking the spec […] As per a sibling comment, it is not breaking the spec. The comment in the Pg code even cites the spec that says what to do (and is quoted in the post): * Generate UUID version 7 per RFC 9562, with the given timestamp. * * UUID version 7 consists of a Unix timestamp in milliseconds (48 * bits) and 74 random bits, excluding the required version and * variant bits. To ensure monotonicity in scenarios of high- * frequency UUID generation, we employ the method "Replace * LeftmostRandom Bits with Increased Clock Precision (Method 3)", * described in the RFC. […]
peterldowns 2y ago
The test should do a set comparison, not an ordered list comparison, if it wants to check that the same 5 accounts were returned by the API. I think it's as simple as that. The blogpost is interesting and I appreciated learning the details of how the UUIDv7 implementation works.
- vips7L 2y ago
  Don’t you think that depends on what you’re guaranteeing in your api? If you’re guaranteeing that your api returns the accounts ordered you need to test for that. But I do agree in general that using a set is the correct move.
  - Too 2y ago
    The test is a very strange example indeed. Is it testing the backend, the database or both? If the api was guaranteeing ordered values, pre-uuid7 the backend must have sorted them by other means before returning, making the test identical. If the backend is not guaranteeing order, that shouldn't be tested either.
paulddraper 2y ago
> Moreover, even if it did, the RFC for UUIDv7 doesn't promise this property. Huh? If the docs were to guarantee it, they guarantee it. Why are you looking for everything to be part of RFC UUIDv7? Failure of logic.
- fwip 2y ago
  Their next sentence explains. Other databases might not make that guarantee, including future versions of Postgres.
throw0101c 2y ago
> […] code that relies on this monotonicity. The reason for this is simple: the documentation doesn't promise this property. Moreover, even if it did, the RFC for UUIDv7 doesn't promise this property. The "RFC for UUIDv7", RFC 9562, explicitly mentions monotonicity in §6.2 ("Monotonicity and Counters"): Monotonicity (each subsequent value being greater than the last) is the backbone of time-based sortable UUIDs. Normally, time-based UUIDs from this document will be monotonic due to an embedded timestamp; however, implementations can guarantee additional monotonicity via the concepts covered in this section. * https://datatracker.ietf.org/doc/html/rfc9562#name-monotonicity-and-counters https://datatracker.ietf.org/doc/html/rfc9562#name-monotonic... In the UUIDv7 definition (§5.7) it explicitly mentions the technique that Postgres employs for rand_a: rand_a: 12 bits of pseudorandom data to provide uniqueness as per Section 6.9 and/or optional constructs to guarantee additional monotonicity as per Section 6.2. Occupies bits 52 through 63 (octets 6-7). * https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-version-7 https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-vers... Note: "optional constructs to guarantee additional monotonicity". Pg makes use of that option.
- stonemetal12 2y ago
  >explicitly mentions monotonicity >optional constructs So it is explicitly mentioned in the RFC as optional, and Pg doesn't state that they guaranty that option. The point still stands, depending on optional behavior is a recipe for failure when the option is no longer taken.
  - idconvict 2y ago
    The "optional" portion is this part of the spec, not the time part. > implementations can guarantee additional monotonicity via the concepts covered in this section
    - dragonwriter 2y ago
      The “time part” is actually two different parts: the required millisecond-level ordering and the optional use of part of rand_a (which postgres does) to provide higher-resolution (nanosecond, in the postgres case) time ordering when combined with the required portion. So, no, the “time part” of the postgres implementation is, in part, one of the options discussed in the spec, not merely the “time part” required in the spec.
braiamp 2y ago
I don't think most people will heed this warning. I warned people in a programming forum that Python ordering of objects by insertion time was a implementation detail, because it's not guaranteed by any PEP [0]. I could literally write a PEP compliant Python interpreter and could blow up in someone's code because they rely on the CPython interpreter behavior. [0]: https://mail.python.org/pipermail/python-dev/2017-December/151283.html https://mail.python.org/pipermail/python-dev/2017-December/1...
- kstrauser 2y ago
  That definitely was true, and I use to jitter my code a little to deliberately find and break tests that depended on any particular ordering. It's now explicitly documented to be true, and you can officially rely on it. From https://docs.python.org/3/library/stdtypes.html#dict https://docs.python.org/3/library/stdtypes.html#dict: > Changed in version 3.7: Dictionary order is guaranteed to be insertion order. That link documents the Python language's semantics, not the behavior of any particular interpreter.
- dragonwriter 2y ago
  > I warned people in a programming forum that Python ordering of objects by insertion time was a implementation detail, because it's not guaranteed by any PEP PEPs do not provide a spec for Python, they neither cover the initial base language before the PEP process started, nor were all subsequent language changes made through PEPs. The closest thing Python has to a cross-implementation standard is the Python Language Reference for a particular version, treating as excluded anything explicitly noted as a CPython implementation detail. Dictionaries being insertion-ordered went from a CPython implementation detail in 3.6 to guaranteed language feature in 3.7+.
sedatk 2y ago
As a counter-argument, it will inevitably turn into a spec if it becomes widely-used enough. What was that saying, like: “every behavior of software eventually becomes API”
- tomstuart 2y ago
  https://www.hyrumslaw.com/ https://www.hyrumslaw.com/
  - sedatk 2y ago
    Yes, that one! Thanks :)
- the8472 2y ago
  Consider the incentives you're setting up there. An API contract goes both ways, the vendor promises some things and not others to preserve flexibility, and the user has to abide by it to not get broken in the future. If you unilaterally ignore the contract, even plan to do so in advance, then eventually kindness and capacity to accommodate such abuse will run might run out and they may switch to an adversarial stance. See QUIC for example which is a big middle finger to middle boxes.
  - sedatk 2y ago
    Sure, there is a risk. But, it all depends on how great and desirable the benefits are.
deadbabe 2y ago
Most code does not live for a long time. Similar to how consumer products are built for planned obsolescence, code is also built with a specific lifespan in mind. If you spend time making code bulletproof so it can run for like 100 years, you will have wasted a lot of effort for nothing when someone comes along and wipes it clean and replaces it with new code in 2 years. Requirements change, code changes, it’s the nature of business. Remember any fool can build a bridge that stands, it takes an engineer to make a bridge that barely stands.
- Pxtl 2y ago
  Uh, more people work on 20-year-old codebases than you'd think.
  - 9dev 2y ago
    And yet these people are dwarved by the number of developers crunching out generic line of business CRUD apps every day.
- agilob 2y ago
  >Most code does not live for a long time. Sure, and here I am in a third company doing cloud migration and changing our default DB from MySQL to SQL server. The pain is real, 2 year long roadmap is now 5 years longer roadmap. All because some dude negotiated a discount on cloud services. And we still develop integrations that talk to systems written for DOS.
- mardifoufs 2y ago
  What? Okay, so assume that most code doesn't last. It doesn't mean that you should purposefully make it brittle for basically no additional benefit? If as you say, it's about making the most with as little as possible (which is what the bridge analogy usually refers to), then surely adding a single function (to actually enforce the ordering you want) to make your code more robust is one of the best examples of that?
drbojingle 2y ago
In enterprise land. In proof of concept land that's not quite true (but does become true if the concept works)
StackTopherFlow 2y ago
I agree, optimizing for readability and maintainability is almost always the right choice.

3eb7988a1663 2y ago

I too am missing the win on this. It is breaking the spec, and does not seem like it offers a significant advantage. In the eventual event where you have a collection of UUID7 you are only ever going to be able to rely on the millisecond precision anyway.

sbuttgereit 2y ago
You say it's breaking the spec, but is it? From https://www.rfc-editor.org/rfc/rfc9562.html#name-uuid-version-7 https://www.rfc-editor.org/rfc/rfc9562.html#name-uuid-versio...: "UUIDv7 values are created by allocating a Unix timestamp in milliseconds in the most significant 48 bits and filling the remaining 74 bits, excluding the required version and variant bits, with random bits for each new UUIDv7 generated to provide uniqueness as per Section 6.9. Alternatively, implementations MAY fill the 74 bits, jointly, with a combination of the following subfields, in this order from the most significant bits to the least, to guarantee additional monotonicity within a millisecond: 1. An OPTIONAL sub-millisecond timestamp fraction (12 bits at maximum) as per Section 6.2 (Method 3). 2. An OPTIONAL carefully seeded counter as per Section 6.2 (Method 1 or 2). 3. Random data for each new UUIDv7 generated for any remaining space." Which the referenced "method 3" is: "Replace Leftmost Random Bits with Increased Clock Precision (Method 3): For UUIDv7, which has millisecond timestamp precision, it is possible to use additional clock precision available on the system to substitute for up to 12 random bits immediately following the timestamp. This can provide values that are time ordered with sub-millisecond precision, using however many bits are appropriate in the implementation environment. With this method, the additional time precision bits MUST follow the timestamp as the next available bit in the rand_a field for UUIDv7."
throw0101c 2y ago
> It is breaking the spec […] As per a sibling comment, it is not breaking the spec. The comment in the Pg code even cites the spec that says what to do (and is quoted in the post): * Generate UUID version 7 per RFC 9562, with the given timestamp. * * UUID version 7 consists of a Unix timestamp in milliseconds (48 * bits) and 74 random bits, excluding the required version and * variant bits. To ensure monotonicity in scenarios of high- * frequency UUID generation, we employ the method "Replace * LeftmostRandom Bits with Increased Clock Precision (Method 3)", * described in the RFC. […]

peterldowns 2y ago

The test should do a set comparison, not an ordered list comparison, if it wants to check that the same 5 accounts were returned by the API. I think it's as simple as that. The blogpost is interesting and I appreciated learning the details of how the UUIDv7 implementation works.

vips7L 2y ago
Don’t you think that depends on what you’re guaranteeing in your api? If you’re guaranteeing that your api returns the accounts ordered you need to test for that. But I do agree in general that using a set is the correct move.
- Too 2y ago
  The test is a very strange example indeed. Is it testing the backend, the database or both? If the api was guaranteeing ordered values, pre-uuid7 the backend must have sorted them by other means before returning, making the test identical. If the backend is not guaranteeing order, that shouldn't be tested either.

paulddraper 2y ago

> Moreover, even if it did, the RFC for UUIDv7 doesn't promise this property. Huh? If the docs were to guarantee it, they guarantee it. Why are you looking for everything to be part of RFC UUIDv7? Failure of logic.

fwip 2y ago
Their next sentence explains. Other databases might not make that guarantee, including future versions of Postgres.

throw0101c 2y ago

> […] code that relies on this monotonicity. The reason for this is simple: the documentation doesn't promise this property. Moreover, even if it did, the RFC for UUIDv7 doesn't promise this property. The "RFC for UUIDv7", RFC 9562, explicitly mentions monotonicity in §6.2 ("Monotonicity and Counters"): Monotonicity (each subsequent value being greater than the last) is the backbone of time-based sortable UUIDs. Normally, time-based UUIDs from this document will be monotonic due to an embedded timestamp; however, implementations can guarantee additional monotonicity via the concepts covered in this section. * https://datatracker.ietf.org/doc/html/rfc9562#name-monotonicity-and-counters https://datatracker.ietf.org/doc/html/rfc9562#name-monotonic... In the UUIDv7 definition (§5.7) it explicitly mentions the technique that Postgres employs for rand_a: rand_a: 12 bits of pseudorandom data to provide uniqueness as per Section 6.9 and/or optional constructs to guarantee additional monotonicity as per Section 6.2. Occupies bits 52 through 63 (octets 6-7). * https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-version-7 https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-vers... Note: "optional constructs to guarantee additional monotonicity". Pg makes use of that option.

stonemetal12 2y ago
>explicitly mentions monotonicity >optional constructs So it is explicitly mentioned in the RFC as optional, and Pg doesn't state that they guaranty that option. The point still stands, depending on optional behavior is a recipe for failure when the option is no longer taken.
- idconvict 2y ago
  The "optional" portion is this part of the spec, not the time part. > implementations can guarantee additional monotonicity via the concepts covered in this section
  - dragonwriter 2y ago
    The “time part” is actually two different parts: the required millisecond-level ordering and the optional use of part of rand_a (which postgres does) to provide higher-resolution (nanosecond, in the postgres case) time ordering when combined with the required portion. So, no, the “time part” of the postgres implementation is, in part, one of the options discussed in the spec, not merely the “time part” required in the spec.

braiamp 2y ago

I don't think most people will heed this warning. I warned people in a programming forum that Python ordering of objects by insertion time was a implementation detail, because it's not guaranteed by any PEP [0]. I could literally write a PEP compliant Python interpreter and could blow up in someone's code because they rely on the CPython interpreter behavior. [0]: https://mail.python.org/pipermail/python-dev/2017-December/151283.html https://mail.python.org/pipermail/python-dev/2017-December/1...

kstrauser 2y ago
That definitely was true, and I use to jitter my code a little to deliberately find and break tests that depended on any particular ordering. It's now explicitly documented to be true, and you can officially rely on it. From https://docs.python.org/3/library/stdtypes.html#dict https://docs.python.org/3/library/stdtypes.html#dict: > Changed in version 3.7: Dictionary order is guaranteed to be insertion order. That link documents the Python language's semantics, not the behavior of any particular interpreter.
dragonwriter 2y ago
> I warned people in a programming forum that Python ordering of objects by insertion time was a implementation detail, because it's not guaranteed by any PEP PEPs do not provide a spec for Python, they neither cover the initial base language before the PEP process started, nor were all subsequent language changes made through PEPs. The closest thing Python has to a cross-implementation standard is the Python Language Reference for a particular version, treating as excluded anything explicitly noted as a CPython implementation detail. Dictionaries being insertion-ordered went from a CPython implementation detail in 3.6 to guaranteed language feature in 3.7+.

sedatk 2y ago

As a counter-argument, it will inevitably turn into a spec if it becomes widely-used enough. What was that saying, like: “every behavior of software eventually becomes API”

tomstuart 2y ago
https://www.hyrumslaw.com/ https://www.hyrumslaw.com/
- sedatk 2y ago
  Yes, that one! Thanks :)
the8472 2y ago
Consider the incentives you're setting up there. An API contract goes both ways, the vendor promises some things and not others to preserve flexibility, and the user has to abide by it to not get broken in the future. If you unilaterally ignore the contract, even plan to do so in advance, then eventually kindness and capacity to accommodate such abuse will run might run out and they may switch to an adversarial stance. See QUIC for example which is a big middle finger to middle boxes.
- sedatk 2y ago
  Sure, there is a risk. But, it all depends on how great and desirable the benefits are.

deadbabe 2y ago

Most code does not live for a long time. Similar to how consumer products are built for planned obsolescence, code is also built with a specific lifespan in mind. If you spend time making code bulletproof so it can run for like 100 years, you will have wasted a lot of effort for nothing when someone comes along and wipes it clean and replaces it with new code in 2 years. Requirements change, code changes, it’s the nature of business. Remember any fool can build a bridge that stands, it takes an engineer to make a bridge that barely stands.

Pxtl 2y ago
Uh, more people work on 20-year-old codebases than you'd think.
- 9dev 2y ago
  And yet these people are dwarved by the number of developers crunching out generic line of business CRUD apps every day.
agilob 2y ago
>Most code does not live for a long time. Sure, and here I am in a third company doing cloud migration and changing our default DB from MySQL to SQL server. The pain is real, 2 year long roadmap is now 5 years longer roadmap. All because some dude negotiated a discount on cloud services. And we still develop integrations that talk to systems written for DOS.
mardifoufs 2y ago
What? Okay, so assume that most code doesn't last. It doesn't mean that you should purposefully make it brittle for basically no additional benefit? If as you say, it's about making the most with as little as possible (which is what the bridge analogy usually refers to), then surely adding a single function (to actually enforce the ordering you want) to make your code more robust is one of the best examples of that?

drbojingle 2y ago

In enterprise land. In proof of concept land that's not quite true (but does become true if the concept works)

StackTopherFlow 2y ago

I agree, optimizing for readability and maintainability is almost always the right choice.