1. 10 Dec, 2019 1 commit
    • Larry Gritz's avatar
      Exif read: guard better against out of range offests (#2429) · ad793890
      Larry Gritz authored
      The dataptr() function returns nullptr if the offset is out of proper
      range. But the array_to_spec function calling dataptr did not check for
      a nullptr return, ended up segfaulting.
      
      This can happen with JPEG files that have certain malformed (we think)
      sections of their camera maker notes.
      ad793890
  2. 08 Dec, 2019 2 commits
  3. 05 Dec, 2019 1 commit
  4. 04 Dec, 2019 4 commits
  5. 03 Dec, 2019 2 commits
    • Larry Gritz's avatar
      Python: add binding for missing ParamValue constructors (#2417) · 64bad0cc
      Larry Gritz authored
      We previously exposed the PV constructors from just a plain int, float,
      or string, but there wasn't an easy way to construct from arbitrary data
      like there is in C++.
      64bad0cc
    • Larry Gritz's avatar
      Update embedded fmtlib to v6.1.0 (#2423) · 47980599
      Larry Gritz authored
      This updated version of fmt has its floating point conversions locale-independent as we wanted.
      
      Switch completely to relying on fmtlib instead of tinyformat by default, even for the printf-style formatting and to_string() implementation. (It's still controlled by a preprocessor symbol so we can switch back if there's an emergency bug.)
      
      Be more careful about %d vs %u string formatting: tinyformat ignored the distinction, fmt does not.
      
      Summary of performance results:
      
      * float->string (printf style and to_string) is almost 10x faster than before, and now is ~3x faster than C++ std::sprintf to buffer.
      
      * int->string is as fast or faster than std::sprintf to a buffer (despite also doing allocation!).
      
      *  A 6-item conversion to string of mixed int/float/string values is 5.6x faster than before when using printf-style, 1.5x faster for fmt-style, and those two now approximately match and are each 2.7x faster than std::sprintf to pre-allocated buffer.
      47980599
  6. 02 Dec, 2019 3 commits
    • Larry Gritz's avatar
      0eaa5f48
    • Larry Gritz's avatar
      Simplify isfinite idiom. (#2422) · caeb255b
      Larry Gritz authored
      This resulted from some post-commit discussion on issue #2301
      caeb255b
    • Larry Gritz's avatar
      IC/TS option "trust_file_extensions" (#2421) · 47c16dea
      Larry Gritz authored
      The default is 0, old behavior. If you set to nonzero, you are
      promising the ImageCache that it will always be able to identify a
      texture file's format based on the filename extension (e.g. ".exr"
      always means it's an openexr file, ".tif" or "tx" for TIFF files, and
      you will NEVER call a file ".tx" if it's really openexr, or if you do,
      you are ok with it being unable to open it). When you enable it with
      that promise, it can eliminate a few redundant file opens and header
      reads, by not needing to verify file format types at certain stages of
      processing.
      
      I'm not sure I would recommend enabling this in general. But if you
      are operating in a closed environment where you are sure that file
      extensions will match their types (such as a studio with rigid naming
      conventions, where all possible textures have been converted with
      maketx), this can eliminate some unnecessary network traffic and file
      server workload.
      47c16dea
  7. 01 Dec, 2019 1 commit
    • Larry Gritz's avatar
      Fix occasional CI failure in imageinout_test (#2420) · e19c47a9
      Larry Gritz authored
      One test is to write an image file to disk, then write the same image
      as a "file" to an in-memory buffer, and make sure resulting bit patterns
      match exactly.
      
      Every once in a while, it would spuriously fail. I'm pretty sure it's
      because of metadata not matching exactly if the two writes happen on
      opposite sides of a minute boundary!
      
      This patch attempts to preempt that by setting "DateTime" metadata to
      a fixed value, so that individual format writers don't, in its absence,
      supply the "now" value, which could vary from run to run.
      e19c47a9
  8. 29 Nov, 2019 1 commit
    • Larry Gritz's avatar
      oiiotool & ImageBuf improvements for huge images (#2414) · 7be26c2e
      Larry Gritz authored
      * Detect failed allocations when ImageBuf tries to allocate local memory
        but there is not enough memory -- catch the exception, leave the IB
        uninitialized, set an error in the IB. This at least allows oiiotool
        to issue an understandable error message if --create or --pattern fail
        because too high a resolution was specified.
      
      * oiiotool --create and --pattern take a new optional parameter:
        `:type=name` that overrides the default behavior of allocating all
        internal buffers as float. An important application of this might be
        if you are creating a huge image that would not usually fit in memory
        as floats, might as a half or uint8. For example,
      
            oiiotool -pattern checker 64000x32000 3 -o big.exr
      
        writes a really big checkerboard, but will fail on a machine that can't
        muster an allocation of 23 GB RAM. However,
      
            oiiotool -pattern:type=uint8 checker 64000x32000 3 -o big.exr
      
        will create and write the same checkerboard using only 6 GB RAM.
      
      * Noticed that oiiotool -o is the only command that used `:datatype=`
        as an optional type override. All others use `type=`, so change to
        that. (The old one is silently accepted as well.)
      7be26c2e
  9. 28 Nov, 2019 2 commits
    • Larry Gritz's avatar
      BMP fix uninitialized data in output (#2418) · 3c8dccd9
      Larry Gritz authored
      Difference between padded scanline width and number of pixel values was
      not properly considered or zeroed out. This was (nondeterministically)
      botching occasional CI tests.
      3c8dccd9
    • Larry Gritz's avatar
      Assertion overhaul (#2411) · 7c48edb0
      Larry Gritz authored
      The new preferred assertion macro, OIIO_ASSERT, tests a condition, and
      if not true, prints an error message and aborts for debug builds. For
      release builds, only tests and reports the error, but does not abort.
      
      OIIO_DASSERT is identical for debug buils, but does nothing at all for
      release builds (not even test the condition).
      
      ASSERT and DASSERT are the old macros, but because of namespace
      pollution and less desirable behavior, we consider them deprecated
      want to phase out their use. (In a separate review we'll convert calls
      to these.)
      
      Moving forward, then, the attitude about assertions should be:
      
      * OIIO_ASSERT for when you want an error message in all cases, and also
        abort for debug builds (but not release builds).
      
      * OIIO_DASSERT when you want error/abort for debug builds, and nothing
        for release builds.
      
      * ASSERT/DASSERT - old names and behavior, phase out over time.
      
      * assert() - roughly equivalent to OIIO_DASSERT, ok to use as well,
        especially if you are desiring to have a particular header or module
        free of dependency on dassert.h.
      
      * Overall: remember that you should only use assertions to test for
        "impossible" conditions that indicate a programming error. Never use
        them in place of error handling, or to detect bad input.
      7c48edb0
  10. 27 Nov, 2019 4 commits
  11. 20 Nov, 2019 2 commits
  12. 19 Nov, 2019 2 commits
    • Larry Gritz's avatar
      Strutil::lstrip, rstrip (#2409) · e25d956e
      Larry Gritz authored
      We already had strip, which trims chars from both sides. Sometimes you
      just want to trip the left or right.
      
      Also add some missing unit tests. And saw a > vs >= improvement in
      in string_view::substr().  (I don't think it was wrong before, just
      failed to take an early out for an empty substring.)
      e25d956e
    • Larry Gritz's avatar
      More care with locales -- Strutil::vsprintf was not locale independent (#2410) · cb4176ea
      Larry Gritz authored
      We relied on vsnprintf underneath, which is locale dependent.  This is
      important for OSL, which generates code that uses this (it will botch
      the formatting of floating point values in OSL printf, format, error,
      and other similar calls, if the user has their system set to a locale
      with commas as decimal separators).
      
      OSX, FreeBSD, and Windows have vsnprintf_l which allows passing a locale
      (for which we use the "C" locale). But Linux does not.
      
      Instead, we use the public domain stb_sprintf, which is locale independent,
      for its vsnprintf implementation.
      
      Also found a spot in rla output that could botch certain metadata when
      in the wrong locale.
      cb4176ea
  13. 18 Nov, 2019 1 commit
    • Larry Gritz's avatar
      Reduce tile cache entry 'used' contention (#2407) · d095f0da
      Larry Gritz authored
      Each texture query may perform many samples, and each sample may fetch a
      tile (or more). EVERY time it was making sure that the 'used' atomic int
      was set to 1. That's potentially a lot of contention.
      
      Idea: Only set it once for all those samples. We really only need to mark
      it 'used' per texture call, not per sample.
      
      So I augmented find_tile() with an optional parameter mark_same_tile_used
      (defaulting to true), and the effect is that if it happens to be the
      most recently used tile in the microcache, it will only call use() if
      mark_same_tile_used is true. So then, when looping over sampling, it
      passes sample==1 for that param, having the effect that for the first
      sample only (or of course if any other tile is needed than the last one)
      will it mark used and diddle the atomic variable.
      
      In our production scene benchmarks at SPI, when run in a highly
      threaded configuration with 52 cores, I'm seeing a 2% average speedup
      in overall rendering time (geometric mean of many tests; on a few
      tests I'm seeing up to 20% gain, which is crazy, but many other scenes
      merely break even).
      d095f0da
  14. 17 Nov, 2019 3 commits
  15. 16 Nov, 2019 3 commits
  16. 15 Nov, 2019 3 commits
    • Larry Gritz's avatar
      error -> errorf (#2393) · f8807ad7
      Larry Gritz authored
      A while back we added errorf() in various places, which henceforth
      will be the place to do printf-style formatting, trying to migrate to
      a point where the "non-f" error() will eventually follow std::format
      (python-like) formatting codes. But for now, error() and errorf() do
      the same thing, we will only change the meaning of error() down the
      road when it's safe.
      
      This patch just rounds up a lot more places in the internals where
      we use error() and changed to errorf().
      f8807ad7
    • Larry Gritz's avatar
    • Larry Gritz's avatar
      sgi: big file safety and speedups (#2402) · 11f845e6
      Larry Gritz authored
      Watch out for int32 overflows, practice 64 bit seeking.
      
      Also speedups by changing allocations from vector to unique_ptr
      11f845e6
  17. 14 Nov, 2019 2 commits
  18. 13 Nov, 2019 3 commits