jas_image.c: shift exponent UB in bitstoint() (followup to inttobits fix)

[JorgeBarredo14]

Describe the bug

bitstoint in src/libjasper/base/jas_image.c uses raw 1 << prec and 1 << (prec - 1) shifts on a signed int, without bounding prec. Its sibling inttobits was patched to use JAS_ONES(prec) but bitstoint was not updated and still has the same UB when prec is 31 or 32.

Current code on master (approx line 1034-1040):

static jas_seqent_t bitstoint(uint_fast32_t v, unsigned prec, bool sgnd)
{
    jas_seqent_t ret;
    v &= JAS_ONES(prec);                                       // patched
    ret = (sgnd && (v & (1 << (prec - 1)))) ? (v - (1 << prec)) : v;  // NOT patched
    return ret;
}

UBSan on a crafted .j2k with prec = 32:

jas_image.c:1044:4: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'

Same class as #283 / #334. The first line was already fixed (JAS_ONES), the second line was missed.

To Reproduce

Attached PoC: poc_jasper005_bitstoint_shift.j2k

Reproduces with a UBSan build of master.

Expected behavior

No UB for any precision the file header can express.

Fix idea

--- a/src/libjasper/base/jas_image.c
+++ b/src/libjasper/base/jas_image.c
@@ -1034,7 +1034,13 @@
 static jas_seqent_t bitstoint(uint_fast32_t v, unsigned prec, bool sgnd)
 {
     jas_seqent_t ret;
+    if (prec == 0 || prec > 32) {
+        return 0;
+    }
     v &= JAS_ONES(prec);
-    ret = (sgnd && (v & (1 << (prec - 1)))) ? (v - (1 << prec)) : v;
+    const uint_fast32_t sign_bit = (uint_fast32_t)1 << (prec - 1);
+    ret = (sgnd && (v & sign_bit))
+        ? (jas_seqent_t)(v - sign_bit - sign_bit)
+        : (jas_seqent_t)v;
     return ret;
 }

sign_bit - sign_bit expresses 1 << prec without the prec=32 overflow. Same clamp of prec belongs wherever prec is first read from the codestream so the decoder bails early instead of returning 0 silently.

PoC is zipped because GitHub blocks .j2k uploads. Unzip to get poc_jasper005_bitstoint_shift.j2k.

poc_jasper005_bitstoint_shift.zip