Read binary encodings: identify the boundaries before the value

Find the byte boundaries before naming the encoding. A successful decode alone proves little: short strings often fit several alphabets. Start with the enclosing file, protocol field, or label. Then check whether each length, continuation bit, and padding character agrees.

Separate three questions: how does text represent bytes, where does each field end, and what does that field mean? Base64 answers the first question. DER answers parts of the second. A certificate schema supplies much of the third. Skipping a layer can produce plausible numbers from the wrong bytes.

How to tell which encoding you have

Use this table to choose your next check. Its clues narrow the possibilities; they do not identify arbitrary bytes conclusively. Preserve the original text before removing separators or padding.

Read LEB128 payloads from the least significant group

Unsigned LEB128 places seven value bits in each byte. The high bit means another byte follows. Give successive payloads weights of 1, 128, 16384, and so on. The WebAssembly integer specification distinguishes unsigned and signed LEB128. This example uses the unsigned form.

Enter e5 8e 26 in the varint decoder. Remove each continuation bit, then add the contributions below. The result is 624485. Stop at the terminating byte; do not consume the next field as another payload group.

Protobuf adds a field key before the value. For 08 96 01, the key is 08. Its low three bits give wire type 0; shifting right three bits gives field 1. The remaining 96 01 gives 150. The protobuf encoding guide documents this exact message.

Wire type zero does not reveal the declared field type. Read the schema before interpreting a signed value. Protobuf sint fields use ZigZag before varint encoding. Signed LEB128 uses a different signed representation.

Read BCD one decimal digit at a time

In unsigned packed 8421 BCD, each nibble holds one decimal digit. A nibble contains four bits. Decimal 13 therefore becomes 0001 0011. Read the two nibbles separately. Treating that byte as an unsigned binary integer instead gives 19.

Reproduce this with the BCD encoder. Select 8 4 2 1, enable packed output, disable signed output, and select Nibbles. Intel's architecture manual, section 4.7 defines BCD digits and packed byte order.

Do not reject every nibble above nine without checking the field definition. Signed packed formats can reserve a nibble for signs. Other BCD schemes also change the digit mapping. A decimal point requires a separate scale convention; these nibbles encode integers.

Count Base32 and Base64 padding without adding bytes

Base32 assigns five bits to each data character; Base64 assigns six. Padding characters complete text groups without adding decoded bytes. RFC 4648, sections 3–6 defines the alphabets, padding rules, and zero pad bits. Omit padding only when the enclosing specification permits it.

In MZXW6===, read five data characters through the Base32 alphabet, then collect complete eight-bit bytes. The groups are 01100 11001 10111 10110 11110. Discard only the unused zero pad bit. The Base32 decoder returns decimal byte values and shows a UTF-8 preview by default. Its byte explanation shows each five-bit group and the bytes accumulated so far.

For Zg==, the Base64 alphabet gives 011001 100000. Keep 01100110 as the byte and discard the remaining zero pad bits. Select Decode in the Base64 encoder and decoder to check the Base64 rows. Turn off Remove non-alphabet chars during diagnosis; Decode enables it by default. Otherwise, a decoder can hide accidental punctuation or a mismatched alphabet. An accepted result does not prove that the original text used a canonical representation.

Separate dump layout from integer byte order

00000000  12 34  |.4|

The left column gives a byte offset. The middle contains two bytes; the right previews characters. The dot represents a nonprinting byte here. Copy only 12 34 into the hex decoder. Copying the offset adds bytes that never belonged to the data. Two hex digits encode one byte, so 64 hex characters hold 32 bytes.

Interpret those bytes as one unsigned 16-bit integer. Big-endian order gives 0x1234, or 4660. Little-endian order gives 0x3412, or 13330. Check either value with the binary and hex calculator, using unsigned 16-bit addition with zero.

Byte order changes each field's interpretation. It does not reverse every byte across an entire file. Find the field width in the format specification before swapping anything. Hexadecimal notation alone specifies neither width nor byte order.

Remove PEM text before reading DER tag-length-value

RFC 7468, section 2 describes labeled boundaries around Base64 data. The label identifies the expected structure. It does not prove that the enclosed bytes satisfy that structure.

-----BEGIN EXAMPLE-----
MAMCAQU=
-----END EXAMPLE-----

This instructional EXAMPLE label names no RFC 7468 key or certificate format. The PEM to hex converter returns 3003020105. Read the following table from offset zero. Each length counts value bytes, excluding that field's tag and length bytes.

ITU X.690, sections 8.1 and 10.1 specifies tag and length encoding. DER requires definite lengths with minimal length encoding. Long-form lengths state how many following bytes encode the value length. Hex conversion alone does not validate nested DER, a key, or a signature.

Read OID subidentifiers in the opposite group order

OIDs also use seven-bit payloads and continuation bits. However, each subidentifier places its most significant group first. Multiply the running value by 128 before adding each new payload. Do not apply the LEB128 contribution order.

Enter value bytes 2a864886f70d in the hex to object identifier converter. It returns 1.2.840.113549. The first subidentifier combines the first two arcs using 40 × first + second. Here, its explanation gives 42 = 40 × 1 + 2. The next subidentifier, 86 48, accumulates 6, then 840.

Decode the combined first subidentifier before separating its arcs. Values below 40 use first arc 0 and second arc equal to the value. Values from 40 through 79 use first arc 1 and second arc equal to the value minus 40. Values of 80 or more use first arc 2 and second arc equal to the value minus 80.

The first subidentifier can span multiple bytes. For 883703, 88 37 gives 8 × 128 + 55 = 1079. Then 1079 − 80 = 999, so the result is 2.999.3. X.690, section 8.19 includes this example. Pass only OID value bytes to this converter. Remove the enclosing ASN.1 tag and length first.