PHP: rfc:pack-unpack-endianness-signed-integers-support

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== PHP RFC: Add pack()/unpack() support for signed integers with specific endianness ======

  * Version: 1.0
  * Date: 2025-09-15
  * Author: Alexandre Daubois, <alexandredaubois@php.net>
  * Status: Draft
  * Implementation: https://github.com/php/php-src/pull/19368

===== Introduction =====

This RFC proposes adding support for signed integers with specific endianness to PHP's <php>pack()</php> and <php>unpack()</php> functions. This addresses GitHub issue #17068 and fixes the format letter choices in the current implementation (PR #19368).

Currently, PHP's pack/unpack functions support:
  * Machine-endian signed integers: <php>s</php>, <php>l</php>, <php>q</php> (2, 4, 8 bytes)
  * Machine-endian unsigned integers: <php>S</php>, <php>L</php>, <php>Q</php> (2, 4, 8 bytes)
  * Endian-specific unsigned integers: <php>v</php>/<php>n</php>, <php>V</php>/<php>N</php>, <php>P</php>/<php>J</php> (2, 4, 8 bytes)

However, there is **no support for signed integers with specific endianness**, forcing developers to use manual workarounds:

<PHP>
<?php
// Current manual approach for signed little-endian 4-byte integer
$unpackToSignedInt = static function (string $v) {
    $unpacked = unpack('va/Cb/cc', $v);
    return ($unpacked['c'] << 24) | ($unpacked['b'] << 16) | $unpacked['a'];
};

// Proposed approach
$value = unpack('w', $binaryData)[1]; // signed little-endian 2-byte
?>
</PHP>

===== Perl Specification Reference =====

According to the Perl documentation (https://perldoc.perl.org/functions/pack), Perl handles signed integers with endianness using modifier syntax:

<code>
s<   signed 16-bit, little-endian byte order
s>   signed 16-bit, big-endian byte order
l<   signed 32-bit, little-endian byte order  
l>   signed 32-bit, big-endian byte order
q<   signed 64-bit, little-endian byte order
q>   signed 64-bit, big-endian byte order
</code>

The Perl documentation states: "Starting with Perl 5.10.0, integer and floating-point formats... may all be followed by the '>' or '<' endianness modifiers to respectively enforce big- or little-endian byte-order."

===== Why Perl's Approach Cannot Be Used in PHP =====

While Perl's specification provides the ideal reference, PHP cannot adopt Perl's exact syntax for several technical reasons:

**1. Base Letters Already Taken**

PHP already uses the base letters for machine-endian signed integers:
  * <php>s</php> = signed 16-bit (machine endian)
  * <php>l</php> = signed 32-bit (machine endian)  
  * <php>q</php> = signed 64-bit (machine endian)

**2. Parser Architecture Limitations**

Perl uses modifier syntax where endianness indicators (<php><</php>, <php>></php>) follow the base format letter. PHP's pack format parser is designed around single-character format codes in switch/case statements, not compound expressions like <php>s<</php> or <php>s></php>.

**3. Different Design Philosophy**

PHP established a pattern of using completely different letters for endian-specific variants:
  * Unsigned endian-specific: <php>v</php>/<php>n</php> (2-byte), <php>V</php>/<php>N</php> (4-byte), <php>P</php>/<php>J</php> (8-byte)
  * Rather than modifiers like Perl's approach

===== Current Implementation Problems =====

The current PR #19368 introduces arbitrary format letters that don't follow any logical pattern:

<code>
m/y  for signed 2-byte (little/big endian)
M/Y  for signed 4-byte (little/big endian)  
p/j  for signed 8-byte (little/big endian)
</code>

**Issues with current choices:**
  * No relationship to Perl's base letters (<php>s</php>, <php>l</php>, <php>q</php>)
  * No logical pairing with existing unsigned endian formats
  * Arbitrary selection that doesn't follow PHP's established patterns

===== Format Letter Analysis =====

**Currently Used Letters:**

Lowercase: <php>a</php>, <php>c</php>, <php>d</php>, <php>e</php>, <php>f</php>, <php>g</php>, <php>h</php>, <php>i</php>, <php>j</php>, <php>l</php>, <php>m</php>, <php>n</php>, <php>p</php>, <php>q</php>, <php>s</php>, <php>v</php>, <php>x</php>, <php>y</php>

Uppercase: <php>A</php>, <php>C</php>, <php>E</php>, <php>G</php>, <php>H</php>, <php>I</php>, <php>J</php>, <php>L</php>, <php>M</php>, <php>N</php>, <php>P</php>, <php>Q</php>, <php>S</php>, <php>V</php>, <php>X</php>, <php>Y</php>, <php>Z</php>

**Available Letters:**

Lowercase: <php>b</php>, <php>k</php>, <php>o</php>, <php>r</php>, <php>t</php>, <php>u</php>, <php>w</php>, <php>z</php>

Uppercase: <php>B</php>, <php>D</php>, <php>F</php>, <php>K</php>, <php>O</php>, <php>R</php>, <php>T</php>, <php>U</php>, <php>W</php>

===== Proposed Solution =====

Replace the current arbitrary letter choices with letters that follow PHP's established conventions and create logical relationships with existing formats:

**Proposed Format Letters:**
  * <php>w</php>/<php>W</php> for signed 2-byte (little/big endian)
  * <php>t</php>/<php>T</php> for signed 4-byte (little/big endian)  
  * <php>r</php>/<php>R</php> for signed 8-byte (little/big endian)

**Rationale:**
  * **Follows PHP convention**: lowercase = little-endian, uppercase = big-endian
  * **Systematic approach**: Creates consistent pairs rather than arbitrary letter choices
  * **Available letters**: All proposed letters are currently unused
  * **Closest to Perl's intent**: While we can't use Perl's exact `s`/`l`/`q` base letters (already taken), these letters provide a systematic alternative

===== Comparison Tables =====

**Perl vs PHP Approaches:**

^ Perl Specification ^ Current PR (Wrong) ^ Proposed Solution ^
| s< (signed 2-byte LE) | m | w |
| s> (signed 2-byte BE) | y | W |
| l< (signed 4-byte LE) | M | t |
| l> (signed 4-byte BE) | Y | T |
| q< (signed 8-byte LE) | p | r |
| q> (signed 8-byte BE) | j | R |

**PHP Format Letter Organization:**

^ Type ^ 2-byte ^ 4-byte ^ 8-byte ^
| Unsigned LE | v | V | P |
| Unsigned BE | n | N | J |
| Signed LE | w (proposed) | t (proposed) | r (proposed) |
| Signed BE | W (proposed) | T (proposed) | R (proposed) |

===== Platform Considerations =====

**32-bit Platform Behavior:**

On 32-bit platforms, 8-byte format codes (<php>r</php>/<php>R</php>) will throw a <php>ValueError</php> with the message "64-bit format codes are not available for 32-bit versions of PHP", consistent with existing behavior for <php>q</php>/<php>Q</php>/<php>P</php>/<php>J</php>.

<PHP>
<?php
// On 32-bit platforms
try {
    pack('r', 1);
} catch (ValueError $e) {
    echo $e->getMessage(); // "64-bit format codes are not available..."
}
?>
</PHP>

===== Backward Incompatible Changes =====

This change modifies the format letters introduced in PR #19368. Since that PR hasn't been released yet, there are no backward compatibility concerns for existing code.

The proposed letters (<php>w</php>, <php>W</php>, <php>t</php>, <php>T</php>, <php>r</php>, <php>R</php>) are currently unused in PHP's pack/unpack implementation.

===== Proposed PHP Version(s) =====

PHP 8.6 (next minor version)

===== Voting Choices =====

<doodle title="Add signed integer endianness support to pack()/unpack() with proposed format letters?" voteType="single" closed="true">
   * Yes
   * No
</doodle>

===== Implementation =====

The implementation is available in PR #19368, which requires updating the format letters from the current arbitrary choices to the proposed systematic approach outlined in this RFC.

Changes required in the pull request if this get accepted:
  * Replace <php>m</php> with <php>w</php>, <php>y</php> with <php>W</php>
  * Replace <php>M</php> with <php>t</php>, <php>Y</php> with <php>T</php>
  * Replace <php>p</php> with <php>r</php>, <php>j</php> with <php>R</php>

===== References =====

  - GitHub Issue: https://github.com/php/php-src/issues/17068
  - Current Implementation: https://github.com/php/php-src/pull/19368  
  - Perl pack documentation: https://perldoc.perl.org/functions/pack
  - PHP pack documentation: https://www.php.net/manual/en/function.pack.php
Page Tools
