HomePhabricator

Avoid implementation-defined and undefined behavior when dealing with sizes

Description

Avoid implementation-defined and undefined behavior when dealing with sizes

Summary:

  • Fix possible integer overflow in DER parsing

If we’re in the last loop iteration, then lenleft == 1 and it could
be the case that ret == MAX_SIZE, and so ret + lenleft will
overflow to 0 and the sanity check will not catch it. Then we will
return (int) MAX_SIZE, which should be avoided because this value is
implementation-defined. (However, this is harmless because
(int) MAX_SIZE == -1 on all supported platforms.)

  • Parse DER-enconded length into a size_t instead of an int

This avoids a possibly implementation-defined signed (int) to unsigned
(size_t) conversion portably.

  • Simplify control flow in DER parsing
  • Avoid out-of-bound pointers and integer overflows in size comparisons

This changes pointer calculations in size comparions to a form that
ensures that no out-of-bound pointers are computed, because even their
computation yields undefined behavior.
Also, this changes size comparions to a form that ensures that neither
the left-hand side nor the right-hand side can overflow.

This is a backport of secp256k1 PR578

Test Plan:

make check

Reviewers: #bitcoin_abc, Fabien

Reviewed By: #bitcoin_abc, Fabien

Subscribers: Fabien

Differential Revision: https://reviews.bitcoinabc.org/D4998

Details

Provenance
Tim Ruffing <crypto@timruffing.de>Authored on Nov 1 2018, 11:15
deadalnixCommitted on Jan 17 2020, 16:07
deadalnixPushed on Jan 17 2020, 16:07
Reviewer
Restricted Project
Differential Revision
D4998: Avoid implementation-defined and undefined behavior when dealing with sizes
Parents
rABCfd526a1dc259: Guard memcmp in tests against mixed size inputs.
Branches
Unknown
Tags
Unknown