add BLAKE3 hash by divinity76 · Pull Request #13194 · php/php-src

divinity76 · 2024-01-19T11:29:46Z

BLAKE3 is a very fast cryptographically secure hash. It is the latest iteration of the BLAKE hash, which was a SHA3 finalist (but lost to Keccak in the final round, for being too similar to SHA2).
RFC: https://wiki.php.net/rfc/blake3
Check this speed chart:

and this /ext/hash/bench.php run where BLAKE3 is much faster than every other secure hash, and beats several non-cryptographic hashes (AMD Ryzen 9 7950x):

$ sapi/cli/php ext/hash/bench.php
crc32b       0.001195
crc32c       0.001202
crc32        0.001202
xxh3         0.001234
xxh128       0.001289
xxh64        0.001475
xxh32        0.002235
murmur3f     0.002459
murmur3c     0.003681
murmur3a     0.004289
adler32      0.007718
blake3       0.007752
md4          0.013109
fnv132       0.015075
fnv164       0.015109
fnv1a64      0.015116
fnv1a32      0.015251
joaat        0.018858
md5          0.019797
sha1         0.020472
tiger160,3   0.021290
tiger192,3   0.021363
tiger128,3   0.021366
tiger128,4   0.027518
tiger160,4   0.027743
tiger192,4   0.027870
ripemd128    0.029190
ripemd256    0.029378
sha3-224     0.029787
sha3-256     0.031518
haval256,3   0.038328
haval224,3   0.038479
haval128,3   0.038483
sha3-384     0.038559
haval192,3   0.038564
haval160,3   0.039068
sha512/256   0.039302
sha512       0.039307
sha512/224   0.039472
sha384       0.039508
ripemd160    0.042287
ripemd320    0.043036
sha3-512     0.054044
haval192,4   0.055699
haval224,4   0.055902
haval160,4   0.055925
haval256,4   0.055948
haval128,4   0.056165
sha256       0.057846
sha224       0.058139
haval128,5   0.070442
haval224,5   0.070503
haval256,5   0.070569
haval192,5   0.070576
haval160,5   0.071109
whirlpool    0.086256
gost         0.200251
gost-crypto  0.200709
snefru256    0.449650
snefru       0.451111
md2          1.237880

Quoting Google Bard:

In summary, BLAKE3 is a modern and highly efficient cryptographic hash function that offers security, flexibility, and ease of use. Its impressive speed and adaptability make it a promising choice for a wide range of applications in the digital age.

on x86_64 BLAKE3 is slightly faster than KangarooTwelve, but on ARM BLAKE3 is significantly faster than KangarooTwelve.
AFAIK the PHP project doesn't like git submodules (for example Dmitry Stogov wanted the new JIT engine to be a submodule but the PHP developers refused), so instead I made ext/hash/blake3/fetch_upstream_blake3.sh
I have not added SSE2/AVX2/AVX512/etc optimized builds to MSVC because I don't have a MSVC system to test on, so I just added the bare-minimum portable implementation to MSVC builds. (feel free to fix it, I don't want to.)

ext/hash/blake3/fetch_upstream_blake3.sh

devnexen · 2024-01-19T11:48:42Z

That s quite an addition ... worth discussing.

requested at php#13194 (comment)

iluuu1994 · 2024-01-19T13:30:22Z

This is indeed a lot of code. I think this should be discussed on the list, and possibly voted on.

divinity76 · 2024-01-19T18:54:30Z

asked on php-internals mailing list: https://marc.info/?l=php-internals&m=170568974400302&w=2
(and fixed MSVC and ARM+Neon builds, the latter tested on Oracle Cloud ARM64, which is notably different from Apple's ARM)

divinity76 · 2024-01-19T19:19:20Z

ext/hash/php_hash_blake3.h

+#define PHP_BLAKE3_CTX blake3_hasher
+// help: is V correct?
+//#define PHP_BLAKE3_SPEC "b8b8qb64bbbbb1760"
+#define PHP_BLAKE3_SPEC "L8L8Qa64CCCCL8Ca1760"


can someone double-check this? because i couldn't wrap my head around it, and this is just my best guess based on

php-src/ext/hash/blake3/upstream_blake3/c/blake3.h

Lines 42 to 61 in 882466b

typedef struct {

uint32_t cv[8];

uint64_t chunk_counter;

uint8_t buf[BLAKE3_BLOCK_LEN];

uint8_t buf_len;

uint8_t blocks_compressed;

uint8_t flags;

} blake3_chunk_state;

typedef struct {

uint32_t key[8];

blake3_chunk_state chunk;

uint8_t cv_stack_len;

// The stack size is MAX_DEPTH + 1 because we do lazy merging. For example,

// with 7 chunks, we have 3 entries in the stack. Adding an 8th chunk

// requires a 4th entry, rather than merging everything down to 1, because we

// don't know whether more input is coming. This is different from how the

// reference implementation does things.

uint8_t cv_stack[(BLAKE3_MAX_DEPTH + 1) * BLAKE3_OUT_LEN];

} blake3_hasher;

How did you come up with this? From what I see in

php-src/ext/hash/hash.c

Lines 211 to 230 in 6d4598e

/* Serialize a hash context according to a `spec` string.

Spec contents:

b[COUNT] -- serialize COUNT bytes

s[COUNT] -- serialize COUNT 16-bit integers

l[COUNT] -- serialize COUNT 32-bit integers

q[COUNT] -- serialize COUNT 64-bit integers

i[COUNT] -- serialize COUNT `int`s

B[COUNT] -- skip COUNT bytes

S[COUNT], L[COUNT], etc. -- uppercase versions skip instead of read

. (must be last character) -- assert that the hash context has exactly

this size

Example: "llllllb64l16." is the spec for an MD5 context: 6 32-bit

integers, followed by 64 bytes, then 16 32-bit integers, and that's

exactly the size of the context.

The serialization result is an array. Each integer is serialized as a

32-bit integer, except that a run of 2 or more bytes is encoded as a

string, and each 64-bit integer is serialized as two 32-bit integers, least

significant bits first. This allows 32-bit and 64-bit architectures to

interchange serialized HashContexts. */

, Q and C is not even there? This is a serialization format so best way to verify is to add some test and try serialization. Around the place that I link is also code for the actual serialization so it should be pretty easy to debug this and figure out.

Look at line 150, uppercase Q still exists:

php-src/ext/hash/hash.c

Line 150 in 6d4598e

} else if (*spec == 'q' || *spec == 'Q') {

Pretty sure uppercase C also existed with the meaning unsigned char or uint8_t when it was written, but it seems to have been removed! I'll try to re-write it again.. thanks!

Still can't get it quite right, do you see anything off here?

#define PHP_BLAKE3_SPEC /* uint32_t key[8]; */"l8" \ /* uint32_t cv[8]; */ "l8" \ /* uint64_t chunk_counter; */ "q1" \ /* uint8_t buf[BLAKE3_BLOCK_LEN]; */ "b64" \ /* uint8_t buf_len; */ "b1" \ /* uint8_t blocks_compressed */ "b1" \ /* uint8_t flags; */ "b1" \ /* uint8_t cv_stack_len; */ "b1" \ /* uint8_t cv_stack[(BLAKE3_MAX_DEPTH + 1) * BLAKE3_OUT_LEN]; */ "b1760" \ "."

It still hits

php-src/ext/hash/hash.c

Lines 1493 to 1495 in 6d4598e

serialize_failure:

zend_throw_exception_ex(NULL, 0, "HashContext for algorithm \"%s\" cannot be serialized", hash->ops->algo);

RETURN_THROWS();

I found a possible solution: it seems the compiler insert 5 alignment-padding-bytes after uint8_t buf_len;,
and an additional 7 padding bytes after uint8_t cv_stack[(BLAKE3_MAX_DEPTH + 1) * BLAKE3_OUT_LEN];, if the spec is changed to

#define PHP_BLAKE3_SPEC /* uint32_t key[8]; */"l8" \ /* uint32_t cv[8]; */ "l8" \ /* uint64_t chunk_counter; */ "q" \ /* uint8_t buf[BLAKE3_BLOCK_LEN]; */ "b64" \ /* uint8_t buf_len; */ "b" \ /* skip 5 bytes of alignment padding in chunk */ "B5" \ /* uint8_t blocks_compressed */ "b" \ /* uint8_t flags; */ "b" \ /* uint8_t cv_stack_len; */ "b" \ /* uint8_t cv_stack[(BLAKE3_MAX_DEPTH + 1) * BLAKE3_OUT_LEN]; */ "b1760" \ /* skip 7 trailing alignment bytes */ "B7" \ "."

serialization actually works on my system!

But this seems like the kind of thing different compilers, and even the same compiler with different optimization settings, may choose to do differently. For example, I wouldn't trust gcc -Ofast (optimize-for-speed) and gcc -Os (optimize-for-size) to do the exact same alignments paddings in the exact same location 🤔 Nor would I trust the compiler to do the exact same alignment padding across 32bit and 64bit builds..

I might be wrong but.. idk!

I think 5 bytes is correct only on 64bit but it should not be correct on 32bit. Also the compiler is free to choose how to do padding so your choice of the padding place is basically relaying on undefined behaviour and it could theoretical differ between compilers. As I mentioned, the only way how to be sure is to define fields as compiler must not re-order them.

From what I see, there's also option to define your own serialization function but not sure if it would make it any simpler.

Well it could be cleaner because it wouldn't rely on struct memory layout...

I think 5 bytes is correct only on 64bit but it should not be correct on 32bit

well 263b2f6 works on 32bit (tested locally on a debian 12 32bit docker image, and now confirmed on the 32bit github CI runner)

From what I see, there's also option to define your own serialization function but not sure if it would make it any simpler.

hmm could you link to an example? I haven't seen it. A custom serializer sounds like the safest bet across compilers/architectures/configurations

well 263b2f6 works on 32bit (tested locally on a debian 12 32bit docker image, and now confirmed on the 32bit github CI runner)

so it pads to 64bit on 32bit? interesting. I thought that it gets padded to 32bit on 32bit arch but probably I'm wrong.

hmm could you link to an example?

You can define your own php_hash_serialize_func_t and php_hash_unserialize_func_t that you can set to php_hash_ops instead of php_hash_serialize / php_hash_unserialize. Then you should be able to do a custom serialization / unserialization and ignore the spec.

ext/hash/blake3/fetch_upstream_blake3.sh

divinity76 · 2024-01-25T06:12:01Z

post on the mailing list worth re-posting here:
GCC11.4, even with -march=native -mtune=native, which is not commonly used in PHP,
the compiler didn't stand a chance against the hand-optimized assembly versions

wrote some benchmarks, but the TL;DR is:
portable -O2 usually used by PHP managed 1126MB/s,
portable -O2 -march=native managed 533MB/s (wtf? gcc obviously got
something wrong here),
hand-written -O2 SSE2 managed 3144MB/s,
hand-written -O2 SSE41 managed 3332MB/s,
hand-written -O2 avx2 managed 6554MB/s,
hand-writen -O2 AVX512 managed 8913MB/s,
on my AMD Ryzen 9 7950x,
benchmarking code:
https://gist.github.com/divinity76/5729472dd5d77e94cd0acb245aac2226
raw output:

array(6) {
  ["O2-portable-march"]=>
  array(2) {
    ["microseconds_for_16_kib"]=>
    int(29295)
    ["mb_per_second"]=>
    float(533.3674688513398)
  }
  ["O2-portable"]=>
  array(2) {
    ["microseconds_for_16_kib"]=>
    int(13876)
    ["mb_per_second"]=>
    float(1126.0449697319111)
  }
  ["O2-sse2"]=>
  array(2) {
    ["microseconds_for_16_kib"]=>
    int(4969)
    ["mb_per_second"]=>
    float(3144.4958744214127)
  }
  ["O2-sse41"]=>
  array(2) {
    ["microseconds_for_16_kib"]=>
    int(4688)
    ["mb_per_second"]=>
    float(3332.977815699659)
  }
  ["O2-avx2"]=>
  array(2) {
    ["microseconds_for_16_kib"]=>
    int(2384)
    ["mb_per_second"]=>
    float(6554.1107382550335)
  }
  ["O2-avx512"]=>
  array(2) {
    ["microseconds_for_16_kib"]=>
    int(1753)
    ["mb_per_second"]=>
    float(8913.291500285226)
  }
}

Edit:
tested ARM Neon optimizations on Oracle Cloud's cheapest ARM VPS:
VM.Standard.A1.Flex, Ubuntu 22.04, GCC11.4,
results:

-O2 portable: 596MB/s
-O2 -march=native portable: 601MB/s
-O2 ARM Neon optimized implementation: 1138MB/s

Again, even with -march=native, the compiler cannot make the portable
implementation nearly as fast as the hand-optimized cpu-specific
implementation.

ext/hash/config.m4

ext/hash/config.w32

ext/hash/blake3/fetch_upstream_blake3.sh

bukka · 2024-02-02T21:14:36Z

It's missing license in each file so that will need to be added.

divinity76 · 2024-02-02T23:21:18Z

@bukka added the LICENSE file from https://github.com/BLAKE3-team/BLAKE3/blob/master/LICENSE ,
but adding them to every file sounds excessive, are you sure we actually want that?

petk · 2024-02-03T10:54:28Z

Regarding the license, can we maybe add it to the README.REDIST.BINS file? Number 21. Blake3 (ext/hash/blake3) and content of LICENSE file copied into it at the end. This file is also distributed when download PHP source code either for Windows or the tarball from downloads.php.net so it's on one place, and code is a bit less cluttered.

bukka · 2024-02-03T13:52:07Z

I think the added LICENSE file and a note in README.REDIST.BINS should be enough for upstream files. You should add a header to the files that you added - hash_blake3.c and php_hash_blake3.h

ext/hash/config.m4

divinity76 · 2024-02-14T08:44:50Z

@bukka @petk added LICENSE headers and reference: 1954b37 - I prefer a reference in README.REDIST.BINS rather than in-lining it.

cypherbits · 2024-04-14T09:20:36Z

Having this would be good

cypherbits · 2024-10-21T18:28:21Z

More than 2 years since the first attempt to get Blake3 into PHP, why is it so hard?

divinity76 · 2024-10-22T07:57:58Z

More than 2 years since the first attempt to get Blake3 into PHP, why is it so hard?

the main problem this round was that I just lost steam. Got occupied with other stuff and forgot about this. You reminded me though!

php#13194 (comment)

divinity76 · 2025-01-22T21:45:28Z

removed controversial fetch_upstream_blake3.sh/patches.diff/verify-generated-files stuff.
updated branch to php8.5-dev master.
updated to upstream blake3 1.5.5
added some -Wall -Werror fixes: silence -Wunused-function on MacOS-arm64 BLAKE3-team/BLAKE3#382 / add missing MAYBE_UNUSED BLAKE3-team/BLAKE3#443
i think the MACOS_X64_DEBUG_NTS failure is not my fault, it appears to be a random timeout error.
(PS would be nice if someone could double-check that PHP_BLAKE3_SPEC looks correct.)

I'm requesting re-review.

remicollet · 2025-01-23T07:30:10Z

-1 from me

I think it is a terrible idea to bundle more crypto stuff in PHP
For such serious things, we should rely on external libraries (OpenSSL, sodium...) provided by project specialized in crypto.
We already have bundled far too much things.

Is there an RFC for this ?

divinity76 · 2025-01-24T09:05:30Z

@remicollet imo we should support BLAKE3 for the same reason we support xxHash (added 8.1.0): speed. BLAKE3 offers SHA3-256-like security at much higher speed than SHA3-256. Quoting /ext/hash/bench.php: sha3-256 0.031518 blake3 0.007752.

Exactly how we support BLAKE3 is not that important, if OpenSSL/Sodium starts offering BLAKE3, I wouldn't mind requiring OpenSSL/Sodium for BLAKE3 support. But they don't support BLAKE3 (yet?).

As for RFC, made a draft a year ago, but now I can't even find it. I'll make a new one.

remicollet · 2025-01-24T09:55:09Z

Exactly how we support BLAKE3 is not that important, if OpenSSL/Sodium starts offering BLAKE3, I wouldn't mind requiring OpenSSL/Sodium for BLAKE3 support. But they don't support BLAKE3 (yet?).

So effort should go there, not here.

divinity76 · 2025-01-24T10:08:45Z

RFC draft: https://wiki.php.net/rfc/blake3

bukka

Except the missing serialization, it seem ok to me. It would be also good to add a bit more tests to test some predefined vectors to be sure all is good.

I think it's good already for the RFC so you can announce it.

In terms of the maintanence, the hash extension bundles already quite a few of algos so this is just another one. It's taken from upstream like some other algorithms so I don't think it matters that much the amount of code. The upstream seems to be quite well maintained.

ext/hash/hash_blake3.c

bukka · 2025-01-24T10:53:01Z

ext/hash/php_hash_blake3.h

+#define PHP_BLAKE3_CTX blake3_hasher
+// help: is V correct?
+//#define PHP_BLAKE3_SPEC "b8b8qb64bbbbb1760"
+#define PHP_BLAKE3_SPEC "L8L8Qa64CCCCL8Ca1760"


How did you come up with this? From what I see in

php-src/ext/hash/hash.c

Lines 211 to 230 in 6d4598e

/* Serialize a hash context according to a `spec` string.

Spec contents:

b[COUNT] -- serialize COUNT bytes

s[COUNT] -- serialize COUNT 16-bit integers

l[COUNT] -- serialize COUNT 32-bit integers

q[COUNT] -- serialize COUNT 64-bit integers

i[COUNT] -- serialize COUNT `int`s

B[COUNT] -- skip COUNT bytes

S[COUNT], L[COUNT], etc. -- uppercase versions skip instead of read

. (must be last character) -- assert that the hash context has exactly

this size

Example: "llllllb64l16." is the spec for an MD5 context: 6 32-bit

integers, followed by 64 bytes, then 16 32-bit integers, and that's

exactly the size of the context.

The serialization result is an array. Each integer is serialized as a

32-bit integer, except that a run of 2 or more bytes is encoded as a

string, and each 64-bit integer is serialized as two 32-bit integers, least

significant bits first. This allows 32-bit and 64-bit architectures to

interchange serialized HashContexts. */

, Q and C is not even there? This is a serialization format so best way to verify is to add some test and try serialization. Around the place that I link is also code for the actual serialization so it should be pretty easy to debug this and figure out.

ext/hash/tests/hash_serialize_001.phpt

bukka · 2025-01-24T11:05:35Z

ext/hash/config.m4

@@ -73,6 +106,8 @@ PHP_INSTALL_HEADERS([ext/hash], m4_normalize([
  php_hash_ripemd.h
  php_hash_sha.h
  php_hash_sha3.h
+  php_hash_blake3.h
+  blake3/upstream_blake3/c/blake3.h


Why is there that extra upstream_blake3 directory. Cannot this go directly to blake3?

the directory separate code written for the php-src integration from the substantial amount of code cloned directly from https://github.com/BLAKE3-team/BLAKE3/

Cannot this go directly to blake3?

It can, but I'd prefer if it didn't.

I don't really understand why it needs extra subdirectory. The blake3 directly contains only upstream_blake3 directory which really doesn't make any sense to me. And you also have c directory which I can see is from upstream. So it should really be just blake3/c .

ext/hash/tests/hash-clone.phpt

ext/hash/tests/hash_copy_001.phpt

ext/hash/tests/hash_hmac_algos.phpt

divinity76 · 2025-01-24T11:52:15Z

Right now there is a problem with https://dev.mysql.com/ causing Windows CI's to fail, specifically https://dev.mysql.com/get/Downloads/MySQL-8.0/mysql-8.0.31-winx64.zip returns HTTP 403 Forbidden.

It's not this PR's fault 🤔

edit: made a dedicated issue for it: #17561

mvorisek · 2025-03-10T16:35:50Z

PHP needs fast hash, can the RFC be moved to voting?

divinity76 · 2025-03-11T16:50:46Z

@mvorisek last time I brought it up on the internals mailing list, they were worried about how the ext/hash API was getting cluttered, how it would increase compile time, how it would be a maintenance burden, and started discussing how less-frequently-used-hashes could be moved into pecl extensions (iirc) ... I dropped out of the conversation rather quickly 😅

mvorisek · 2025-03-11T17:07:58Z

Hmm, I cannot comment on the compile times, but given this hash algo is currently the best one given its entropy, cryptographic safety and speed, I would say it should be integrated into the core and made available on every platform.

The maintanance should be also minimal given the source can be verified using CI and thus only the integration needs to be maintained mainly.

Given these, I belive there is enough rationale to integrate this PR. What about simply opening the voting with a possible optional question if the CI sources verification should be added as well. If the voting would pass, great, if not, we cannot do much other than pecl..

bukka · 2025-03-11T18:32:02Z

It's hard to say if those comments would mean that the RFC cannot pass. I don't think it will be much more extra maintenance than other hashes as they are just taken from upstream. In this case the upstream is actually quite lively compare to others. The deciding factor should be also what's good for our users. So I would try to at least put it to vote and see the result then. I think it's worth a try for sure.

TimWolla · 2025-03-11T18:47:23Z

@divinity76 It would certainly help if you'd update the benchmarks to include the latest SHA-2 changes. At least the SSE2 variant should be available everywhere. So it would probably make sense to list SHA-2 separately with the SSE2 implementation and the SHA-NI implementation that should be supported by your CPU.

TimWolla · 2025-03-11T18:50:26Z

And looking back at the thread, you should also specify which of the Blake2 implementations is being used (SSE2, SSE4.1, AVX2, AVX512), because otherwise it's not an apples to apples comparison.

maksam07 · 2025-08-23T10:59:22Z

It's been so long, why isn't the algorithm built into PHP yet? What problems remain?

ndossche · 2025-08-23T13:01:12Z

It's been so long, why isn't the algorithm built into PHP yet? What problems remain?

Because an RFC was requested but not yet brought to discussion+vote

squash and rebase https://github.com/php/php-src/tree/896351a76d138071a21fea1ba20e91d5d0859c1b Co-authored-by: divinity76 <hans@loltek.net> Co-authored-by: Peter Kokot <peterkokot@gmail.com> Co-authored-by: Jakub Zelenka <bukka@php.net>

there's padding issues, idk how portable this is... The CI's should be interesting.

I don't have a 32bit system to test on locally, so testing on CI for now..

Also BLAKE3-team/BLAKE3#382 and BLAKE3-team/BLAKE3#443 have both been accepted upstream, so we can cleanly use upstream BLAKE3 sources without php-specific patches, at least for now :)

github-actions bot added Category: Build System Extension: hash labels Jan 19, 2024

mvorisek reviewed Jan 19, 2024

View reviewed changes

ext/hash/blake3/fetch_upstream_blake3.sh Outdated Show resolved Hide resolved

divinity76 added a commit to divinity76/php-src that referenced this pull request Jan 19, 2024

update github verify-generated-files

fcc16d3

requested at php#13194 (comment)

divinity76 requested review from TimWolla and iluuu1994 as code owners January 19, 2024 12:37

divinity76 force-pushed the hash-blake3 branch from f9f7291 to bf8a527 Compare January 19, 2024 12:54

divinity76 commented Jan 19, 2024

View reviewed changes

divinity76 mentioned this pull request Jan 20, 2024

BLAKE3 hash support #6358

Closed

devnexen reviewed Jan 20, 2024

View reviewed changes

ext/hash/blake3/fetch_upstream_blake3.sh Outdated Show resolved Hide resolved

This was referenced Jan 25, 2024

c++ compatibility hack BLAKE3-team/BLAKE3#377

Closed

blake3.c:354:22: error: logical ‘and’ of mutually exclusive tests is always false [-Werror=logical-op] BLAKE3-team/BLAKE3#379

Closed

petk reviewed Jan 29, 2024

View reviewed changes

ext/hash/config.m4 Outdated Show resolved Hide resolved

petk reviewed Jan 29, 2024

View reviewed changes

ext/hash/config.w32 Outdated Show resolved Hide resolved

petk reviewed Jan 30, 2024

View reviewed changes

ext/hash/blake3/fetch_upstream_blake3.sh Outdated Show resolved Hide resolved

petk reviewed Feb 3, 2024

View reviewed changes

ext/hash/config.m4 Outdated Show resolved Hide resolved

petk reviewed Feb 3, 2024

View reviewed changes

ext/hash/config.m4 Outdated Show resolved Hide resolved

divinity76 mentioned this pull request Feb 8, 2024

segfault in Zend/tests/assign_coalesce_001.phpt #13323

Open

divinity76 added a commit to divinity76/php-src that referenced this pull request Jan 22, 2025

remove controversial fetch_upstream_blake3.sh

b3e063e

php#13194 (comment)

divinity76 requested review from devnexen, mvorisek and petk January 22, 2025 22:36

bukka reviewed Jan 24, 2025

View reviewed changes

mvorisek mentioned this pull request Mar 10, 2025

refactor: Tokenizer hash metode using xxHash PHP-CS-Fixer/PHP-CS-Fixer#8491

Merged

bukka mentioned this pull request Nov 20, 2025

ext/hash: optional support for the crc-fast library #20513

Open

mvorisek mentioned this pull request Mar 14, 2026

Verify BLAKE3 bundled files using CI divinity76/php-src#1

Merged

mvorisek and others added 6 commits March 15, 2026 12:33

initial blake3 commit

b3f6092

squash and rebase https://github.com/php/php-src/tree/896351a76d138071a21fea1ba20e91d5d0859c1b Co-authored-by: divinity76 <hans@loltek.net> Co-authored-by: Peter Kokot <peterkokot@gmail.com> Co-authored-by: Jakub Zelenka <bukka@php.net>

try to fix serialization

1817aaa

there's padding issues, idk how portable this is... The CI's should be interesting.

serialization still broken on 32bit

558b9c5

attempt 32bit serialization fix

f70934f

I don't have a 32bit system to test on locally, so testing on CI for now..

Sync upstream blake3 (latest 1.8.3 release)

7b79e3a

Also BLAKE3-team/BLAKE3#382 and BLAKE3-team/BLAKE3#443 have both been accepted upstream, so we can cleanly use upstream BLAKE3 sources without php-specific patches, at least for now :)

restore padding fix for 32bit

0aa5c88

mvorisek force-pushed the hash-blake3 branch from c8cf681 to 0aa5c88 Compare March 15, 2026 11:39

Verify BLAKE3 bundled files using CI

03ac67e

	typedef struct {
	uint32_t cv[8];
	uint64_t chunk_counter;
	uint8_t buf[BLAKE3_BLOCK_LEN];
	uint8_t buf_len;
	uint8_t blocks_compressed;
	uint8_t flags;
	} blake3_chunk_state;

	typedef struct {
	uint32_t key[8];
	blake3_chunk_state chunk;
	uint8_t cv_stack_len;
	// The stack size is MAX_DEPTH + 1 because we do lazy merging. For example,
	// with 7 chunks, we have 3 entries in the stack. Adding an 8th chunk
	// requires a 4th entry, rather than merging everything down to 1, because we
	// don't know whether more input is coming. This is different from how the
	// reference implementation does things.
	uint8_t cv_stack[(BLAKE3_MAX_DEPTH + 1) * BLAKE3_OUT_LEN];
	} blake3_hasher;

	/* Serialize a hash context according to a `spec` string.
	Spec contents:
	b[COUNT] -- serialize COUNT bytes
	s[COUNT] -- serialize COUNT 16-bit integers
	l[COUNT] -- serialize COUNT 32-bit integers
	q[COUNT] -- serialize COUNT 64-bit integers
	i[COUNT] -- serialize COUNT `int`s
	B[COUNT] -- skip COUNT bytes
	S[COUNT], L[COUNT], etc. -- uppercase versions skip instead of read
	. (must be last character) -- assert that the hash context has exactly
	this size
	Example: "llllllb64l16." is the spec for an MD5 context: 6 32-bit
	integers, followed by 64 bytes, then 16 32-bit integers, and that's
	exactly the size of the context.

	The serialization result is an array. Each integer is serialized as a
	32-bit integer, except that a run of 2 or more bytes is encoded as a
	string, and each 64-bit integer is serialized as two 32-bit integers, least
	significant bits first. This allows 32-bit and 64-bit architectures to
	interchange serialized HashContexts. */

	serialize_failure:
	zend_throw_exception_ex(NULL, 0, "HashContext for algorithm \"%s\" cannot be serialized", hash->ops->algo);
	RETURN_THROWS();

Conversation

divinity76 commented Jan 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

devnexen commented Jan 19, 2024

Uh oh!

iluuu1994 commented Jan 19, 2024

Uh oh!

divinity76 commented Jan 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

divinity76 Jan 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

divinity76 Jan 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

divinity76 Jan 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bukka Jan 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

divinity76 commented Jan 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bukka commented Feb 2, 2024

Uh oh!

divinity76 commented Feb 2, 2024

Uh oh!

petk commented Feb 3, 2024

Uh oh!

bukka commented Feb 3, 2024

Uh oh!

Uh oh!

Uh oh!

divinity76 commented Feb 14, 2024

Uh oh!

cypherbits commented Apr 14, 2024

Uh oh!

cypherbits commented Oct 21, 2024

Uh oh!

divinity76 commented Oct 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

divinity76 commented Jan 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

remicollet commented Jan 23, 2025

Uh oh!

divinity76 commented Jan 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

remicollet commented Jan 24, 2025

Uh oh!

divinity76 commented Jan 24, 2025

Uh oh!

bukka left a comment

Choose a reason for hiding this comment

divinity76 commented Jan 19, 2024 •

edited

Loading

divinity76 commented Jan 19, 2024 •

edited

Loading

divinity76 Jan 19, 2024 •

edited

Loading

divinity76 Jan 24, 2025 •

edited

Loading

divinity76 Jan 24, 2025 •

edited

Loading

bukka Jan 24, 2025 •

edited

Loading

divinity76 commented Jan 25, 2024 •

edited

Loading

divinity76 commented Oct 22, 2024 •

edited

Loading

divinity76 commented Jan 22, 2025 •

edited

Loading

divinity76 commented Jan 24, 2025 •

edited

Loading

divinity76 Jan 24, 2025 •

edited

Loading

divinity76 commented Jan 24, 2025 •

edited

Loading

divinity76 commented Mar 11, 2025 •

edited

Loading