

        <cite>Bryce</cite>
      <time>0:00</time>
      <p>
          It   says   there&#39;s   a   comment.   Slash   slash.   We   must   prepare   for   his   coming.   Everything&#39;s   lowercase   except   for   the   H   and   his.   And   then   destruct   goeser.   And   it   uh   it&#39;s   a   function   object   that   calls   something&#39;s   uh   uh   destructor.
      </p>

        <cite>Jared Hoberock</cite>
      <time>0:21</time>
      <p>
          Well,   the   comment   made   more   sense   because   I   think   that   used   to   be   a   forward   declaration.   So   obviously   this   is   a   Ghostbusters   reference.
      </p>

        <cite>Conor</cite>
      <time>0:42</time>
      <p>
          Welcome   to   AD   SP   the   Podcast,   episode   242,   recorded   on   May   21st,   2025.   My   name   is   Connor,   and   today   with   my   co-host   Bryce,   we   finish   part   four   of   our   four-part   conversation   with   Jared   Hoberock.   We   chat   about   thrust,   parallel   algorithms,   and   more.
      </p>

        <cite>Bryce</cite>
      <time>1:00</time>
      <p>
          That   actually   kind   of   leads   to   another   question   that   I   had   meant   to   ask   earlier,   which   is   um   you   talked   a   little   bit   about   the   transfer   of   thrust   from   uh   from   research   to   production   uh   uh   in   the   handoff,   but   that   was   sort   of   after   Thrust   had   uh   you   know   sort   of   made   it,   you   know?   And   and   uh   you   know,   at   the   time   there   were   a   couple   other   libraries   that   were   kind   of   similar   to   Thrust,   like   uh   CUDPP,   and   there   I   think   there   was   like   one   or   two   other   ones.   And   I   just   wonder,   like,   can   you   tell   us   a   little   bit   about   when   when   you   guys   first   put   Thrust   out   there?   Like,   what   was   the   response?   Was   it   like   uh   did   it   just   very   rapidly   get   a   lot   of   uh   adoption   uh   and   and   grow   in   popularity?   And   like   what   why   was   it   that   thrust   is   the   one   that   that   succeeded   and   became   so   ubiquitous?   And   like   how   quickly   did   that   happen?
      </p>

        <cite>Jared Hoberock</cite>
      <time>2:00</time>
      <p>
          Um   I   mean   the   reason   it   succeeded   is   because   we   chose   the   right   interface.   Uh   how   fast   it   happened,   um   I   don&#39;t   really   remember.   I   I   remember   we   gave   uh   at   the   first   GTC   we   gave   like   two   talks   on   Thrust,   and   I   remember   the   the   crowd   being   large   and   enthusiastic.   Um   so   it   at   least   it   succeeded   enough   to   attract   some   you   know   some   people   to   a   talk.   Do   you   have   the   slides   from   those   talks?   Uh   somewhere.   They   might   be   in   are   they   not   in   the   repository?   Uh   they   might   not   be,   I   don&#39;t   know.   Oh,   I   lost   it.   If   you   look   through   Perforce,   you&#39;ll   find   them.   Um   or   I   might   have   them   in   uh   like   a   Google   Drive   or   something   like   that.   Anyway,   yeah,   they   were   Nathan   gave   a   talk   and   I   gave   a   talk   at   the   GTC.   And   uh   Nathan&#39;s   talk   was   sort   of   here&#39;s   what   thrust   is,   and   I   think   my   talk   was   like   here   to   do,   here&#39;s   how   to   do   some   you   know   interesting   tricks   with   thrust,   and   it&#39;s   just   you   know   walk   the   slides,   just   walk   through   some   of   the   example   programs.
      </p>

        <cite>Bryce</cite>
      <time>3:04</time>
      <p>
          But   uh   so   how   how   m   how   long   uh   so   like   after   that   uh   a   did   you   ever   reach   a   point   where   you   guys   were   just   like   swamped   with   PRs   and   and   like   when   did   when   did   you   when   did   it   become   like   visible   to   you   that   there   were   a   lot   of   people   using   this?   Uh   and   when   did   like   uh   when   when   did   the   writing   was   the   writing   on   the   wall   that   like   it   was   gonna   have   to   be   taken   over   by   the   engineering   team?
      </p>

        <cite>Jared Hoberock</cite>
      <time>3:33</time>
      <p>
          It   never   became   visible   to   me   that   a   lot   of   people   were   using   it.   I   would   have   been   happy   to   uh   I   Nathan   was   really   pro-productization   because   uh   you   know,   for   career   reasons,   I   would   have   rather   not   productize   it   so   we   could   continue   developing   it   in   um   interesting   directions,   I   guess.   I   guess   the   problem   is   when   it   got   productized,   it&#39;s   became   less   flexible.   Like   changing   it   became   harder.   Um   harder.   You   know,   before   we   actually   released   it   on   Google   Code,   like   GitHub   didn&#39;t   even   exist   at   the   time.   So   we&#39;re   the   the   the   initial   release   was   on   Google   Code,   and   if   we   wanted   to   change   something,   we   would   just   change   something.   And   uh   if   people   had   a   bug,   we   would   fix   it   as   you   know,   next   day   or   something   like   that.   Uh   you   can&#39;t   really   do   that   with   a   product.   Um   but   um   I   don&#39;t   know,   like   hearing   you   guys   talk   about   it   uh   or   hearing   you   guys   talk   about   um   people   out   in   the   wild   using   it,   I   guess,   is   when   it   dawned   on   me   that   it   has   a   lot   of   users.   Like   at   the   time,   it   just   seemed   like   uh   you   know   a   project   we   were   working   on   and   we&#39;re   happy   to   work   on.
      </p>

        <cite>Bryce</cite>
      <time>4:41</time>
      <p>
          Yeah,   I   was   I   I   was   I   sort   of   always   had   that   sense,   and   I   mean   I   guess   it&#39;s   it&#39;s   hard   for   folks   who   sit   and   research   because   you   you   produce   a   thing,   and   then   it   if   it   becomes   marginally   successful,   then   it   gets   picked   up   by,   you   know,   some   engineering   team,   and   you   then   you   go   work   on   the   next   thing,   and   you   don&#39;t   necessarily   see   what   the   I   think   it   works   differently   now.
      </p>

        <cite>Jared Hoberock</cite>
      <time>5:07</time>
      <p>
          So   Thrust   was   one   of   the   first   things   that   NV   research   productized,   uh   and   we   didn&#39;t   have   a   lot   of   experience   with   handing   off   things   to   product   teams.   So   I   think   if   it   were   to   happen   today,   uh   researchers   could   be   involved   in   the   product   like   to   the   extent   that   they&#39;re   interested   in.   But   in   those   days,   it   was   sort   of   seen   as   like,   okay,   you   guys   productize   this   thing,   it&#39;s   time   to   work   on   something   something   new.   Um   But   I   think   um   since   then   people   probably   see   the   value   of   keeping   the   original   author   sort   of   in   the   loop,   uh,   maybe   a   bit   more,   if   they   if   they   want   to   be,   you   know,   with   with   uh   more   recent   projects.   Are   you   proud   of   Thrust?   Uh   I   have   mixed   feelings   about   it.   Like   uh   I   guess   what   I   see   is   the   sort   of   missed   opportunities.   Um   and   um   I   sort   of   see   the   the   mistakes   that   could   be   corrected   or   should   have   been   corrected,   but   um   it&#39;s   I   mean   it&#39;s   gratifying   to   see   that   people   still   like   to   use   it   and   think   it&#39;s   it&#39;s   uh   worthwhile.   But   um   it&#39;s   hard   not   to   focus   on   things   you   wish   could   have   been   done   better.
      </p>

        <cite>Bryce</cite>
      <time>6:17</time>
      <p>
          One   one   frequent   one   frequent   thing   that   gets   brought   up   is   while   thrust   is   great   for   many   things   and   it&#39;s   a   great   place   to   start,   if   you&#39;re   doing   if   you&#39;re   in   vo   if   your   algorithm   ends   up   being   one   call,   it&#39;s   a   single   pass   thing,   thrust   is   perfect   for   that.   If   you&#39;re   launching   multiple   things   because   Thrust   doesn&#39;t   have   an   asynchronous   model,   because   it   blocks,   it   can   become   a   performance   bottleneck.   That&#39;s   something   that   uh   uh   is   uh   an   often   cited   criticism   uh   of   Thrust.   Um   and   uh   we   sometimes   think   of   it   as   being   an   on-ramp   uh   for   parallelism   that   uh   it&#39;s   something   it&#39;s   easy   and   gets   people   started.   But   oftentimes   if   you   need   to   architect   a   much   more   complex   application   where   you&#39;re   launching   a   bunch   of   different   kernels,   then   you   have   to   go   and   do   something   a   little   bit   uh   lower   level.   And   uh   uh   had   Thrust   evolved   uh   earlier   and   more   to   support   uh   asynchrony   um   in   a   more   structured   way,   um   uh   perhaps   it   could   have   been   even   more   successful   than   it   is   today.
      </p>

        <cite>Jared Hoberock</cite>
      <time>7:24</time>
      <p>
          Oh,   for   sure.   Uh   and   and   thrust   predates   asynchrony   in   CUDA.   Um   thrust   predates   CUDA   streams,   right?   So   uh   and   it   was   always   not   clear   how   to   how   to   make   it   work   well   with   CUDA   streams,   and   I   don&#39;t   think   it   ever   really   did   uh   or   does   uh   work   well   with   CUDA   streams.   I   guess   um   I   guess   another   regret   is   that   um   in   addition   to   thrust,   we   also   have   Cub,   and   these   two   things   are   have   interdependencies,   like   they&#39;re   cyclic   dependencies   between   the   two   libraries,   uh   which   causes   problems.   I   guess   it&#39;s   it&#39;s   just   a   shame   that   we   couldn&#39;t   find   a   way   to   uh   do   cub-like   things   and   in   thrust   and   or   you   know,   thrust-like   things   in   Cub.   It&#39;s   a   it&#39;s   sort   of   an   awkward   situation   to   have   both   libraries.
      </p>

        <cite>Bryce</cite>
      <time>8:20</time>
      <p>
          Well,   and   it&#39;s   actually   interesting   that   you   you   mentioned   that   because   you   know   we   um   so   when   I   started   at   NVIDIA   in   2017,   uh   I   I   was   Thrust   had   been   essentially   without   a   maintainer   for   like   two   or   three   years.   Uh   and   like   on   day   one,   I   didn&#39;t   even   know   what   I   was   being   hired   to   do   when   I   started.   Uh   like   it   was   unclear   from   the   job   description.   Um,   but   uh   in   and   like   I   my   hiring   was   weird,   but   just   like   Olivier   had   talked   to   someone   and   been   like,   hey,   you   should   interview   this   guy,   and   then   I   interviewed,   and   then   they&#39;re   like,   Oh,   you   want   a   job?   And   they   uh   give   me   an   offer,   and   then   I   showed   up,   and   on   day   one,   they&#39;re   like,   So   here&#39;s   this   thing   thrust,   you&#39;re   now   in   charge   of   it.   There&#39;s   no   product   manager,   nobody&#39;s   really   touched   it   in   two   years,   figure   out   what   to   do   with   it.   Um   and   so   then   I   started   the   uh   the   CUDA   C   core   libraries   team   at   Nvidia,   and   originally   it   was   just   me,   and   then   it   was   me   and   one   other   person   for   some   period   of   time,   and   we   productized   uh   Cub   uh   uh   uh   just   as   Thrust   had   been   productized   and   did   some   amount   of   work   to   uh   unify   things   between   the   two.   But   now,   you   know,   eight   years   later,   that   team,   the   CUDA   C   ⁇   Core   Libraries   team,   uh,   or   I   think   it&#39;s   now   just   the   core   compute   libraries   team,   same   acronym,   but   uh   but   different   meaning.   It&#39;s   now   like   a   I   don&#39;t   know,   it&#39;s   more   than   10   people   team.   It   could   be   15   people   or   so.   It&#39;s   uh   it&#39;s   it&#39;s   a   lot   of   people   working   on   it,   and   they   work   on   Thrust,   Cub,   and   LibQ,   which   is   our   port   of   uh   uh   of   libclus.   And   there&#39;s   a   move   for   all   three   of   these   libraries   to   essentially   uh   merge   together   into   one   unified   thing.   Um   and   they&#39;re   all   living   now   a   monorepo   now,   and   and   in   fact,   there&#39;s   some   there&#39;s   some   talk   about   uh   the   uh   not   the   thrust   name.   The   thrust   name   will   essentially   always   exist   because   there&#39;s   so   many   people   who   have   written   code   for   it,   but   uh   uh   of   migrating   the   thrust   backend   and   sort   of   merging   it   with   some   of   the   other   things.   So   that   the   thrust   code   base,   parts   of   the   thrust   code   base   that   have   been   around   for   a   long   time   now,   essentially   going   away   in   the   future.   Um   and   all   of   these   things   merging   together   into   a   uh   uh   uh   you   know   a   more   uh   unified   uh   uh   structure   uh   down   the   road.   What   uh   I   think   the   last   thing   I   I   would   uh   uh   I&#39;d   ask   you   about   is   um   uh   the   thrust   was   uh   you   know   open   source   sort   of   from   the   start.   Um   and   uh   uh   you   know   like   what   what   was   what   was   the   what   was   that   like?   Um   I   mean   this   was   in   a   very   you   know   an   early   time.   It   was   before   you   know   before   GitHub.   Um   you   mentioned   it   was   like   Google   Code,   but   like   can   you   how   did   you   collaborate   with   people   back   in   the   day?   Like   how   did   how   did   users   find   you?   How   did   you   interact?   Like   what   was   that   like?
      </p>

        <cite>Jared Hoberock</cite>
      <time>11:24</time>
      <p>
          Uh   we   had   a   Google   groups   uh   mailing   list   uh   that   people   would   ask,   how   do   I   do   this?   Uh   in   thrust,   that   sort   of   thing,   or   maybe   they   would   report   bugs.   Um   there   wasn&#39;t   like   we   didn&#39;t   use   Discord   or   Slack   or   anything   like   that.   Um   so   just   through   um   you   know   mailing   lists,   I   guess   that&#39;s   how   people   used   to   collaborate   in   the   2000s,   you   know,   uh   through   a   mailing   list   or   something   like   that.   I   remember   when   we   uh   pushed   the   code   for   the   first   time,   we   we   announced   it   on   Hacker   News   and   uh   some   subreddit.   Um   and   just   you   know   wait   wait   waited   to   see   what   would   happen.   But   um   as   far   as   I   know,   Thrust   was   the   first   open   source   thing   that   uh   NVIDIA   ever   published.   I   don&#39;t   know   of   any   that   are   there.   There   might   be   a   few   um   previous   previous   uh   publications,   but   I   don&#39;t   know   of   them.   Um   so   that   was   um   you   know   we   had   to   get   special   permission   for   that.
      </p>

        <cite>Bryce</cite>
      <time>12:34</time>
      <p>
          You   are   you   are   in   fact   correct   um   that   thrust   was   in   fact   the   first   open   source   project   at   Nvidia,   and   and   uh   uh   we   know   this   because   uh   Thrust   completely   predated   any   open   source   process   at   uh   at   NVIDIA.   And   and   every   like   year   or   two,   somebody&#39;s   like,   hey,   like   why   isn&#39;t   there   like   an   entry   in   like   our   listing   of   open   source   things   like   in   this   database   for   Thrust?   And   we&#39;re   like,   well,   because   it   was   the   first   one.   Like   it   existed   before   all   these   things.
      </p>

        <cite>Jared Hoberock</cite>
      <time>13:07</time>
      <p>
          Yeah,   well,   just   the   nature   of   the   software   uh   demanded   it   that   the   code   be   available,   right?   Because   it&#39;s   uh   C   templates,   so   there&#39;s   no   way   to   hide   it,   and   there&#39;s   no   way   to   use   it   unless   you   have   the   source.   So   uh   had   to   be,   had   to   be   open   source.
      </p>

        <cite>Bryce</cite>
      <time>13:26</time>
      <p>
          Yeah.   And   I   mean,   you   know,   you   mentioned   that   um   thrust   uh   lifted   from   other   projects,   and   in   fact,   thrust   contains,   you   know,   thrust   iterator   code   was   just   taken   from   Boost,   and   some   other   parts   were   taken   from   some   other   places,   and   it   probably   would   have   not   evolved   as   quickly   if   you   had   not   uh   been   able   to   pull   in   all   this   other   stuff.
      </p>

        <cite>Jared Hoberock</cite>
      <time>13:48</time>
      <p>
          That&#39;s   right.   Yeah.   Uh   I   I&#39;m   not   uh   I&#39;m   not   a   expert   GPU   programmer.   Uh   I   can   just   kind   of   get   by.   Uh   but   I&#39;m   good   at   taking   other   people&#39;s   code   and   making   sense   of   it,   so   that&#39;s   uh   mostly   what   we   did.
      </p>

        <cite>Bryce</cite>
      <time>14:05</time>
      <p>
          Uh   Jared,   if   you&#39;re   not   an   expert   GPU   programmer,   I   I   don&#39;t   know   what   we   are.
      </p>

        <cite>Jared Hoberock</cite>
      <time>14:11</time>
      <p>
          There   are   levels.
      </p>

        <cite>Conor</cite>
      <time>14:12</time>
      <p>
          There   are   levels   and   uh   Yeah.   What   does   that   mean   for   us,   Jared?   What   does   that   mean   for   the   people?
      </p>

        <cite>Jared Hoberock</cite>
      <time>14:19</time>
      <p>
          Well,   the   whole   idea   of   thrust   is   that   you   don&#39;t   have   to   be   an   expert   programmer   to   use   a   GPU.   You   just   take   something   off   the   shelf   uh   that   someone   else   has   already   figured   out.
      </p>

        <cite>Bryce</cite>
      <time>14:30</time>
      <p>
          It&#39;s   it&#39;s   really   the   what   was   what   was   the   comrade   tagline?   Uh   the   original   comrade   tagline.
      </p>

        <cite>Jared Hoberock</cite>
      <time>14:38</time>
      <p>
          Own   the   means   of   computation.
      </p>

        <cite>Bryce</cite>
      <time>14:41</time>
      <p>
          It   was   actually   very   fitting.   I   uh   uh   I   when   I   worked   it   uh   on   the   HBX   project,   um   uh   the   original   name   for   the   programming   model   that   Thomas   Sterling   wanted   to   use   was   he   wanted   to   call   it   Agincourt,   uh,   for   the   Battle   of   Agincourt,   which   was   a   uh   I   don&#39;t   know   what   century   battle   between   the   the   British   and   the   French.   And   the   French   had   a   an   army   that   was   primarily   like   mounted   knights,   and   the   English   army   was   primarily   uh   you   know   peasant   longbowmen   and   they   defeated   the   nobility.   And   so   Thomas   Sterling   had   this   had   it   like   this   uh   uh   this   very   idyllic   uh   uh   story   of   this.   It   was   this   programming   model   where   tons   of   tiny   little   cores   and   and   tiny   little   tasks   uh   would   take   over   and   and   topple   the   the   great   evil   uh   that   was   uh   uh   MPI   and   uh   and   and   uh   sort   of   synchronous   message   passing   uh   programming   models.   There   were   there   was   uh   uh   you   know   uh   I   couldn&#39;t   find   it.   I   asked,   I   tried   to   ask   JetGPT   to   search   for   it   while   we   were   chatting,   but   there   were   a   couple   interesting   snippets   of   code   from   like   the   early   days   of   Thrust   that   had   some   quirky   comments   that   if   I   ever   come   across   again,   we&#39;ll   have   to   have   you   back   on   to   ask   you   to   explain.
      </p>

        <cite>Jared Hoberock</cite>
      <time>16:06</time>
      <p>
          Uh   what   are   you   what   what   are   you   thinking   of?
      </p>

        <cite>Bryce</cite>
      <time>16:09</time>
      <p>
          It   was   like   something   like   Frobenator   or   something   like   that.   There   was   some   like   weirdly   named   class   somewhere,   and   then   there   were   a   couple   of   things.
      </p>

        <cite>Jared Hoberock</cite>
      <time>16:17</time>
      <p>
          I   think   I   know   what   you&#39;re   talking   about.   You&#39;re   talking   about   you   talking   about   Gozer?
      </p>

        <cite>Bryce</cite>
      <time>16:22</time>
      <p>
          Yes. I   am   talking   about   Gozer.   What   was   Gozer?
      </p>

        <cite>Jared Hoberock</cite>
      <time>16:28</time>
      <p>
          Uh   is   the   code   still   there   or   is   it   gone?
      </p>

        <cite>Bryce</cite>
      <time>16:31</time>
      <p>
          I&#39;m   looking   right   now.   I   really,   if   they   deleted   that,   I&#39;m   gonna   I&#39;m   gonna   have   some   choice   words.   Oh   no,   I   think   it&#39;s   gone.   How   it   how   did   you   spell   it?   Was   it   one   O   or   two   O&#39;s?
      </p>

        <cite>Jared Hoberock</cite>
      <time>16:41</time>
      <p>
          Gozer   is   spelled   G-O-Z-G-R.   I&#39;m   sure   it&#39;s   in   the   GitHub   history   somewhere.
      </p>

        <cite>Bryce</cite>
      <time>16:50</time>
      <p>
          No,   it&#39;s   still   here.   So   could   you   explain   yourself?   It   says,   I&#39;m   gonna   read   it.   I   think   it&#39;s   pretty   it   says   there&#39;s   a   comment   slash   slash,   we   must   prepare   for   his   coming.   Everything&#39;s   lower   scale   lowercase   except   for   the   H   and   his,   and   then   struct   gozer,   and   it   uh   it&#39;s   a   function   object   that   calls   something&#39;s   uh   uh   destructor.
      </p>

        <cite>Jared Hoberock</cite>
      <time>17:19</time>
      <p>
          Well,   the   comment   made   more   sense   because   I   think   that   used   to   be   a   forward   declaration.   Uh   so   obviously   this   is   a   Ghostbusters   reference.
      </p>

        <cite>Conor</cite>
      <time>17:32</time>
      <p>
          Oh   man.   Obviously,   I   was   born   in   1990.   I   don&#39;t   know   what   this   means.
      </p>

        <cite>Jared Hoberock</cite>
      <time>17:38</time>
      <p>
          Ghost   of   the   destructor   is   the   reference.   So   this   is   a   functor   that   calls   a   destructor.   I   remember   uh   I   got   an   email   uh   some   a   long   time   ago.   I   never   thought   anyone   would   see   this.   I   just   thought   it   was   being   funny,   but   um,   I   got   an   email   uh   from   a   compiler   engineer   at   NVIDIA   because   uh   I   don&#39;t   know   something   about   Thrust   was   exposing   a   compiler   bug   and   it   involved   this   this   uh   struck   that   we&#39;re   looking   at   here.   And   uh   JD   like   posts   the   repro   in   in   the   uh   in   the   email   and   it&#39;s   got   Gozer   uh   staring   back   at   me   and   I&#39;m   like   we   didn&#39;t   we   didn&#39;t   like   discuss   the   name   or   it   or   uh   or   it   I   never   thought   this   would   come   back   to   haunt   me,   but   um   Gozer   is   terrifying   like   that.
      </p>

        <cite>Bryce</cite>
      <time>18:33</time>
      <p>
          At   least   as   long   as   I&#39;m   at   NVIDIA,   I&#39;m   gonna   make   sure   that   uh   this   particular   piece   of   code   remains   in   production.
      </p>

        <cite>Conor</cite>
      <time>18:41</time>
      <p>
          The   real   question   is,   Bryce,   did   you   know   what   this   meant?
      </p>

        <cite>Bryce</cite>
      <time>18:45</time>
      <p>
          I   had   no   idea.   I   am   I   like   you.   I&#39;m   the   same   age   as   you,   Connor.
      </p>

        <cite>Conor</cite>
      <time>18:50</time>
      <p>
          I   mean,   you&#39;re   one   year   you&#39;re   uh   one   year   younger?
      </p>

        <cite>Bryce</cite>
      <time>18:54</time>
      <p>
          Older,   younger?   What   year   were   you   born,   Connor?
      </p>

        <cite>Conor</cite>
      <time>18:57</time>
      <p>
          I&#39;m   90.   You&#39;re   89.   You&#39;re   91,   right?   Yeah.   91,   yeah,   you&#39;re   one   year   younger.   I   was   gonna   say,   I   would   feel   very   bad   about   myself   if   you   understood   the   reference   and   I   did   not.   I   read   this   and   I   was   like,   I   don&#39;t   understand   the   joke,   but   uh   here   we   are.   I   don&#39;t   know   how   many   people   we   have   listening.
      </p>

        <cite>Bryce</cite>
      <time>19:15</time>
      <p>
          I   think   over   time   it&#39;ll   roughly   be   three   or   four   thousand,   so   I   mean   uh   it   is   it   it&#39;s   line   102   in   thrust   slash   detail   slash   allocator   slash   destroyrange   dot   in   L.
      </p>

        <cite>Jared Hoberock</cite>
      <time>19:30</time>
      <p>
          I&#39;m   sure   you&#39;ve   got   some   boomers   in   your   audience.
      </p>

        <cite>Bryce</cite>
      <time>19:33</time>
      <p>
          Jared,   were   these   uh   I&#39;ve   I&#39;ve   always   wondered   this,   so   I&#39;ll   so   I&#39;ll   I&#39;ll   ask,   even   though   we&#39;re   you   know   over   time,   but   but   why   not?   Um   uh   two   things   I&#39;ll   ask.   One,   uh   Thrust   has   a   lot   of   these   headers   that   are   dot   in   L,   like   inline.   Um   was   this   because   you   at   some   point   had   people   that   were   like   it   would   include   these   in   their   like   .cpp   files   and   you   wanted   to   separate   them   from   the   declarations,   or   was   there   at   some   point   a   built,   like   a   version   of   Thrust   that   had   uh   like   a   mode   where   you   just   like   Boost   Asio   where   you   could   build   like   a   library   version   of   it   in   addition   to   the   header-only   version?   Or   like   what   is   what   did   that   dot   inl   like   naming   convention   mean?
      </p>

        <cite>Jared Hoberock</cite>
      <time>20:18</time>
      <p>
          I   think   that&#39;s   just   a   Jaredism.   I   don&#39;t   remember   why   I   chose   to   do   that.
      </p>

        <cite>Bryce</cite>
      <time>20:25</time>
      <p>
          And   uh   I   I   g   the   the   other   question   is   uh   Thrust   uh   uses   um   dot   h   for   its   header   files,   and   uh   some   other   CUDA   projects   use   like   .kuh   and   some   other   extensions.   Uh   what   like   why   why   doths?   Why   not   uh.kuh?   Or   why   not   make   up   your   own   uh   extension?
      </p>

        <cite>Jared Hoberock</cite>
      <time>20:48</time>
      <p>
          I&#39;ve   always   uh   thought   that   I&#39;ve   always   disliked   dot   cuh.   Uh   but   uh   at   least   at   some   point   there   was   a   goal   that   you   could   include   any   thrust   header   with   a   normal   C   compiler   and   it   wouldn&#39;t   freak   out.
      </p>

        <cite>Bryce</cite>
      <time>21:06</time>
      <p>
          And   um   I   don&#39;t   know   if   that&#39;s   still   the   case,   but   uh   there   wasn&#39;t   you   don&#39;t   you   didn&#39;t   have   to   include   it   with   uh   NVCC,   so   it   seemed   unnecessary   to   that&#39;s   actually   uh   it&#39;s   a   good   point   that   we   we   didn&#39;t   talk   about   this,   but   Thrust   has   this   back-end   system   that   we   we   mentioned,   but   we   didn&#39;t   talk   explicitly   about   the   fact   that   uh   Thrust   can   like   can   target   CPUs,   it   can   target   CPU   parallelism.   Uh   and   it&#39;s   it&#39;s   designed   in   a   way   where   like   no   part   of   the   thrust   interface   says   CUDA,   and   Thrust   has   this   uh   uh   you   know,   it   uses   the   the   host   and   device   terminology   um   uh   where   it&#39;s   very   like   abstract   from   the   GPU.   Like   you   can   have   a   device   backend   that&#39;s   not   a   GPU   backend.   Um   why   why   was   that   uh   like   was   that   intended   to   help   adoption?   Like   why   why   do   that?   Because   it   seems   like   it   created   a   bunch   of   work   too.
      </p>

        <cite>Jared Hoberock</cite>
      <time>22:00</time>
      <p>
          Uh   because   it   was   helpful   um   when   just   debugging   your   program   for   correctness   to   be   able   to   use   like   a   sequential   backend   or   uh   I   don&#39;t   know   like   Open   MP   backend,   for   example.   Um   so   I   think   it   definitely   helped   with   adoption.   Um   just   getting   the   CUDA   stuff   right   in   your   CUDA   C   ⁇   program   is   like   a   task   all   on   its   own.   And   really,   if   you   wanted   to,   you   could   write   everything   uh   you   know,   just   get   your   algorithms   uh   skeleton   together   first   and   make   making   sure   that   it   works   um   in   parallel   before   actually   targeting   uh   a   CUDA   GPU   for   real.   I   mean   you   could   totally   I   could   totally   see   that   uh   being   helpful   to   people.   Like   going   all   in   on   um   GPU   parallelism   from   the   get-go   is   often   a   big   big   first   step.   So   it&#39;s   helpful   to   phase   things   sometimes.   And   I   think   the   CPU   backends   uh   can   help   with   that.   What   was   debugging   like   in   the   early   days?   Uh   well   we   used   to   have   device   emulation   in   CUDA.   So   MVCC,   NVCC   used   to   have   a   CPU   backend.   You   could   put   it   into   a   mode   where   it   would   generate   um   x86   code,   and   you   could   actually   step   through   your   code   in   a   debugger.   So   that&#39;s   what   debugging   was   like   in   the   early   days.   But   then   they   phased   they   phased   that   out   very   soon.   They   used   to   use   an   entirely   different   uh   compiler   backend   for   NVCC.   Um   I   think   it   was   Open64,   I   think   is   what   they   used.   Um   anyway,   I   think   maybe   that   made   that   device   simulation   uh   mode   um   easier.   But   um   I   don&#39;t   know,   I&#39;ve   never   used   debuggers.   I&#39;ve   always   used   printf   debugging.   So   that&#39;s   what   it   was   like   for   me.   Um   Connor   has   some   thoughts.
      </p>

        <cite>Conor</cite>
      <time>23:53</time>
      <p>
          You&#39;ve   always   used   printf   debugging?   You&#39;ve   never   used   debuggers?
      </p>

        <cite>Jared Hoberock</cite>
      <time>23:56</time>
      <p>
          I   used   uh   when   I   used   to   use   uh   Microsoft   um   IDE,   I   used   to   have   a   you   know   a   pretty   good   debugger,   and   I   would   use   it   then,   but   um   no,   these   days   I   don&#39;t   use   uh   IDE.   I   use   printf   and   uh   the   terminal   to   program.
      </p>

        <cite>Bryce</cite>
      <time>24:12</time>
      <p>
          I   also   use   printf.
      </p>

        <cite>Conor</cite>
      <time>24:14</time>
      <p>
          Is   that   like   because   you   watched   The   Last   of   Us   season   one   episode   four   where   it   was   like   survivalist   technology   and   you   really   identified   with   that   and   you   want   it   to   be   like   modern   technology   is   not   for   me   and   I&#39;m   gonna   bunker   down?
      </p>

        <cite>Jared Hoberock</cite>
      <time>24:28</time>
      <p>
          Like   there   used   to   be   this   this   guy   at   NVIDIA,   uh   I   think   he   was   a   dev   tech,   he   had   a   blog,   um,   but   he   was   an   actual   survivalist.   He   wouldn&#39;t   use   libc.   He   would   uhc.   He   wouldn&#39;t   use   any   headers   at   all.   Like   he   would   uh   just   insert   sys   uh   system   calls   if   he   wanted   to   print   into   his   program.   I   guess   because   it   made   his   programs   fast.   I&#39;m   not   actually   sure   why,   but   one   thing   is   it   made   you   know   compiling   and   linking   his   his   program   very   fast.   But   uh   if   he   needed   to   debug,   I   think   he   would   like   put   us   to   the   screen   and   he   would   get   at   put   us   from   a   syscall   or   something   like   that.   So   you   can   there&#39;s   levels   to   survivalism   too.
      </p>

        <cite>Conor</cite>
      <time>25:17</time>
      <p>
          So   you&#39;re   saying   y   you&#39;re   not   even   close   to   uh   what   is   really   possible   in   terms   of   coding   survivalism   using   not   debugging   and   just   using   printf   is   just   the   start.   You&#39;re   not   even   at   the   true   level   of   survivalism   in   coding.
      </p>

        <cite>Jared Hoberock</cite>
      <time>25:33</time>
      <p>
          I   guess   I   guess   uh   I&#39;ve   benefited   from   the   things   I&#39;ve   had   to   program   in   the   past   of   really   well-defined   uh   interfaces   and   uh   like   think   of   every   thrust   algorithm,   like   the   post   conditions   are   spelled   out   explicitly   what   they   are,   so   it   makes   it   really   easy   to   write   unit   tests   for   what   you&#39;re   doing.
      </p>

        <cite>Bryce</cite>
      <time>25:54</time>
      <p>
          Um   Jared,   when   when   you   encounter   a   bug   that   you   have   to   debug,   is   it   typically   a   like   a   parallelism   bug?
      </p>

        <cite>Jared Hoberock</cite>
      <time>26:04</time>
      <p>
          No,   it&#39;s   typically   a   compiler   bug.   Those   are   the   bugs   I   run   into.
      </p>

        <cite>Bryce</cite>
      <time>26:09</time>
      <p>
          When   you   run   into   a   bug   where   you   have   to   put   a   printf   into   your   program,   is   it   usually   something   that&#39;s   like   parallel?
      </p>

        <cite>Jared Hoberock</cite>
      <time>26:15</time>
      <p>
          I   don&#39;t   know   if   that   I   don&#39;t   know   if   I   would   say   that   or   not.   Um   bugs   can   come   from   you   know   bugs   can   come   from   any   reason.   I   will   say   that   now   that   I   I   spend   most   days   in   Rust,   uh   I   don&#39;t   spend   any   time   debugging.   But   when   I   am   writing   um   writing   like   MLIR   code   and   C,   the   first   thing   I   have   to   do   is   debug   whatever   I   wrote   because   it&#39;s   not   gonna   work   the   first   time,   for   sure.   And   then   yeah,   I   I   do   use   printf.
      </p>

        <cite>Bryce</cite>
      <time>26:44</time>
      <p>
          I   usually   use   printf   because   when   I&#39;m   debugging   something,   it&#39;s   usually   something   that   uh   uh   like   if   it&#39;s   if   it&#39;s   something   that&#39;s   not   a   parallel   bug,   it&#39;s   usually   pretty   quick   for   me   to   get   to   the   bottom   of   it.   Um   but   for   the   most   part,   if   it&#39;s   a   bug,   it&#39;s   a   bug   that   is   happening   due   to   some   like   race   condition   or   you   know   synchronization   you   know   thing.   And   so   the   what   I   my   my   process   is   first   I   run   compute   sanitizer   uh   or   like   like   uh   you   know   a   mem   checking   tool   that   tells   me   whether   I&#39;m   randomly   writing   somewhere   to   memory.   And   if   that&#39;s   the   case,   I   get   to   the   bottom   of   that.   And   then   if   that   isn&#39;t   the   case,   then   I   usually   start   adding   printfts   to   to   see   if   I&#39;ve   like   got   some   incorrect   assumption.   And   if   I   open   it   up   in   a   debugger,   usually   the   the   race   condition,   like   the   timing   has   changed.   Um   and   so   it&#39;s   not   particularly   useful.   And   if   I   if   I   go,   if   I   have   to   go   and   build   the   code   in   like   with   debug   symbols   and   like   with   debugging   in   a   way   that   it   would   be   useful   to   do   anything   in   the   debugger,   like   it&#39;s   not   like   whatever   the   timing   condition   is   that   caused   the   problem   has   typically   gone   away.
      </p>

        <cite>Jared Hoberock</cite>
      <time>27:56</time>
      <p>
          One   of   the   first   jobs   I   had   at   NVIDIA,   uh   I   was   working   on   the   um   optics   team.   So   optics   is   like   a   ray   tracing   um   framework.   And   so   one   of   the   first   jobs   I   had   uh   was   to   sit   and   babysit   um   an   emulator   waiting   for   a   race   condition   in   the   chip   that   um   some   ray   tracing   code   exposed.   And   you   just   sit   down   there   for   several   hours   just   waiting   for   something   to   happen   or   not.   Uh   but   that&#39;s   what   I   spent   a   week   doing   uh   shortly   after   coming   to   NVIDIA.   Race   conditions   are   tricky.
      </p>

        <cite>Bryce</cite>
      <time>28:32</time>
      <p>
          I   had   a   I   had   I   had   a   colleague   who   one   time   I   was   uh   I   was   at   the   office,   it   was   like   7   p.m.   and   he&#39;s   sitting   across   from   me,   and   um   he   uh   he   like   stands   up   very   abruptly   and   uh   I&#39;m   like   where   are   you   going?   He&#39;s   like,   I   have   to   go   like   I   have   to   go   monitor   this   bring-up   machine   in   some   other   building.   Um   and   I&#39;m   like,   well,   why   can&#39;t   you   monitor   it   from   here?   And   he&#39;s   like,   you   don&#39;t   understand.   It   can   catch   fire   at   any   moment.   And   so   I   I   later   came   to   understand   that   uh   when   you&#39;re   on   bring-up   duty,   like   one   of   the   jobs   is   like   you   just   go   and   sit   next   to   the   bring-up   machine   and   there&#39;s   a   fire   extinguisher   there   because   like   it&#39;s   it&#39;s   uh   it&#39;s   got   these   giant   you   know   heatsinks   and   fans,   and   nobody&#39;s   figured   out   how   like   what   clock   speed   you   can   run   this   thing   at   safely   yet.   So   sometimes   like   bad   things   might   happen   and   it   might   overheat   and   it   might   spark   and   catch   on   fire.   But   uh   we   figure   out   all   those   bugs   before   they   uh   uh   you   know   get   into   uh   gamers&#39;   hands.
      </p>

        <cite>Jared Hoberock</cite>
      <time>29:39</time>
      <p>
          Yeah,   exactly.
      </p>

        <cite>Bryce</cite>
      <time>29:42</time>
      <p>
          Alright,   uh   I   think   that&#39;s   uh   that&#39;s   all   I   got.   Connor,   you   got   anything   else   you   want   to   ask,   Jared?
      </p>

        <cite>Conor</cite>
      <time>29:47</time>
      <p>
          I   mean,   uh   we&#39;ve   kept   him   past   30   minutes   past   we   said   we   were   gonna   interview   him.   And   uh   I   mean   I   got   my   sequence   question   answered,   I   got   my   counting   iterator   question   answered.   I   feel   like   we   got   a   lot   of   questions   answered.   I   mean,   we&#39;re   gonna   have   to   bring   you   on   in   the   future.   We   talked,   we   alluded   to   if   you   were   gonna   start   from   scratch   today.   And   uh,   you   know,   we   do   work   in   research,   so   you&#39;re   probably   working   on   something.   And   um   at   the   point   in   time   where   we&#39;re   working   on   2.0,   not   of   thrust   necessarily,   but   of   your   next   project,   we   will   definitely   have   to   have   you   back   on.
      </p>

        <cite>Bryce</cite>
      <time>30:28</time>
      <p>
          Um,   but   maybe   that&#39;s   a   good   thing   to   close   with,   which   is   Jared,   do   you   wanna   you   got   anything   you   want   to   plug?   You   wanna   you   wanna   say   anything   or   anything?
      </p>

        <cite>Conor</cite>
      <time>30:38</time>
      <p>
          It&#39;s   not   that   you   wanna   plug.   Do   you   have   anything   that   you   can   plug?   And   if   you   can&#39;t   plug,   what&#39;s   the   general   direction   of   which   you   are   working   on   things?
      </p>

        <cite>Jared Hoberock</cite>
      <time>30:48</time>
      <p>
          So   I   think   I   mentioned   like   uh   I&#39;ve   kind   of   a   wish   list   for   uh   a   programming   language   that   would   make   software   like   Thrust   more   uh   much   easier   to   write   and   maintain.   So   I   would   like   to   try   to   do   something   uh   in   that   space.   And   um   hopefully   that   will   be   successful.   Um   we&#39;ll   see.   Um   if   people   are   interested,   they   can   just   look   at   what&#39;s   going   on   on   my   GitHub   or   not   going   on   there.   Um   but   there   are   some   hints   there   of   what   might   be   next.
      </p>

        <cite>Conor</cite>
      <time>31:24</time>
      <p>
          Some   hints   on   Jared&#39;s   GitHub.   Link   in   the   show   notes,   folks.
      </p>

        <cite>Bryce</cite>
      <time>31:30</time>
      <p>
          Go   and   drop   some   stars.
      </p>

        <cite>Conor</cite>
      <time>31:33</time>
      <p>
          Here&#39;s   the   real   question.   If   I   really   do   release   this   as   like   a   two   and   a   half   hour   episode,   you   should   probably   not   do   that.   I   sh   I   mean   I   shouldn&#39;t.   But   I   feel   like   if   I   if   I   if   I   titled   this   episode,   I   mean   I   won&#39;t,   but   I   could   title   it   NVIDIA   engineers   discuss   like   CUDA   programming,   given   the   zeitgeist.   Do   you   know   the   term   that   I   heard   over   the   last   week?   It&#39;s   called   Gen   Sanity.   Have   you   heard   that?
      </p>

        <cite>Bryce</cite>
      <time>32:01</time>
      <p>
          I   have   not,   but   I   love   it.
      </p>

        <cite>Conor</cite>
      <time>32:03</time>
      <p>
          He   went   to   Taiwan   and   he   gave   his   keynote   at   Computech   on   Sunday   night,   and   on   Monday,   Tuesday,   all   these   shows   were   talking   about   Gen   Sanity   because   he   announced   not   just   that   we   were   launching   our   global   headquarters   in   Taipei,   but   that   also   we   opened   some   platform   to   third-party   vendors   or   something.
      </p>

        <cite>Bryce</cite>
      <time>32:27</time>
      <p>
          Yeah,   NVLink.
      </p>

        <cite>Conor</cite>
      <time>32:30</time>
      <p>
          I&#39;m   an   engineer.   I   I   don&#39;t   really   understand   what&#39;s   going   on,   but   the   point   is,   is   the   MSNBCs   of   the   world   and   the   Bloombergs   of   the   world,   they   love   us   folks.   And   so   if   we   named   if   we   called   this   podcast   three   NVIDIA   engineers   discuss   the   parallel   algorithms   that   power   the   world,   it   would   blow   up.   I&#39;m   not   gonna   do   that,   but   I   I   could.
      </p>

        <cite>Bryce</cite>
      <time>32:53</time>
      <p>
          Oh,   but   we   gotta   ask,   we   gotta   ask   Jared,   what&#39;s   your   favorite   algorithm?
      </p>

        <cite>Conor</cite>
      <time>32:57</time>
      <p>
          Oh,   yeah,   yeah,   yeah.
      </p>

        <cite>Bryce</cite>
      <time>32:59</time>
      <p>
          My   favorite   algorithm?
      </p>

        <cite>Jared Hoberock</cite>
      <time>33:01</time>
      <p>
          Yeah.
      </p>

        <cite>Conor</cite>
      <time>33:02</time>
      <p>
          Inside   of   thrust   and   outside   of   thrust,   if   they&#39;re   a   different   algorithm.
      </p>

        <cite>Jared Hoberock</cite>
      <time>33:07</time>
      <p>
          Uh   um   metropolis,   uh   Metropolis   Light   Transport   is   my   favorite   algorithm,   outside   of   thrust.
      </p>

        <cite>Conor</cite>
      <time>33:15</time>
      <p>
          What   was   that?
      </p>

        <cite>Jared Hoberock</cite>
      <time>33:16</time>
      <p>
          Metropolis   Metropolis   Light   Transport.
      </p>

        <cite>Conor</cite>
      <time>33:21</time>
      <p>
          Metropolis   Light   Transport.   Is   that   sounds   like   something   from   altered   carbon   like   the   Netflix   show.
      </p>

        <cite>Jared Hoberock</cite>
      <time>33:28</time>
      <p>
          No,   it&#39;s   it&#39;s   something   from   uh   graphics,   computer   graphics.
      </p>

        <cite>Bryce</cite>
      <time>33:32</time>
      <p>
          Is   that   like   a   global   illumination   thing?
      </p>

        <cite>Jared Hoberock</cite>
      <time>33:36</time>
      <p>
          Yeah,   it&#39;s   like   a   global   illumination   thing.   It   uh   applies   um   metropolis   hastings   to   uh   to   uh   to   light   transport.   Light   transport   is   just   what   they   call   um   rendering.   It&#39;s   a   fancy   term   for   rendering.
      </p>

        <cite>Bryce</cite>
      <time>33:50</time>
      <p>
          You   can   tell   you   can   tell   someone&#39;s   an   old   timer   at   Nvidia   if   they   don&#39;t   currently   work   in   graphics,   but   have   like   deep   graphics   chops.
      </p>

        <cite>Conor</cite>
      <time>34:00</time>
      <p>
          Be   sure   to   check   these   show   notes   either   in   your   podcast   app   or   at   adspthepodcast.com   for   links   to   anything   we   mentioned   in   today&#39;s   episode,   as   well   as   a   link   to   a   GitHub   discussion   where   you   can   leave   thoughts,   comments,   and   questions.   Thanks   for   listening.   We   hope   you   enjoyed   and   have   a   great   day.
      </p>

        <cite>Bryce</cite>
      <time>34:13</time>
      <p>
          Low   quality,   high   quantity.   That   is   the   tagline   of   our   podcast.
      </p>

        <cite>Conor</cite>
      <time>34:19</time>
      <p>
          That&#39;s   not   the   tagline.   Our   tagline   is   chaos   with   sprinkles   of   information.
      </p>

        <cite>Bryce</cite>
      <time>34:23</time>
      <p>
          All   right.   All   right,   we   should   let   we   should   let   uh   Jared   go.
      </p>

        <cite>Jared Hoberock</cite>
      <time>34:27</time>
      <p>
          The   green   grass   in   the   background   is   gone,   folks.
      </p>

        <cite>Conor</cite>
      <time>34:30</time>
      <p>
          Whether   this   was   one   episode   or   five   episodes,   it   started   out   lush   and   green   in   the   background,   and   now   it   is   a   dark   sky   in   the   background   of   Kansas.   We   appreciate   Jared   for   taking   all   this   time   tonight   to   chat   with   us.   We   will   definitely   have   you   back   in   the   future.   I   think   we   gotta   have   more   NVIDIA   internal   folks.   We   should   get   Dwayne   Merrill   on,   we   should   get   Chris   Check   on,   we   should   get   a   bunch   of   these   folks   on.   I   we&#39;re   a   podcast   hosted   by   two   NVIDIA   engineers.   We   should   have   more   NVIDIA   folks.
      </p>

        <cite>Bryce</cite>
      <time>35:01</time>
      <p>
          No,   I   will   I   will   I   will   say   very   clearly   that   this   is   not   a   work   activity.   This   is   something   we   do   outside   of   work.
      </p>

        <cite>Conor</cite>
      <time>35:10</time>
      <p>
          I   mean,   yes,   I   I   that   was   not   clear   from   what   I   just   said,   so   it   should   be   clear   now.   But   the   point   being   is   we   do   have   access   to   folks   that   work   at   NVIDIA,   hence   why   Jared   is   on   the   podcast.
      </p>

        <cite>Bryce</cite>
      <time>35:22</time>
      <p>
          Let&#39;s   ask   Jared,   Jared,   who   should   we   interview   next?   Oh   yeah,   that&#39;s   a   good   question.
      </p>

        <cite>Jared Hoberock</cite>
      <time>35:29</time>
      <p>
          Ian   Buck.   Do   you   think   Ian   would   come   on   the   podcast?   Uh   second   question   to   Jared.   Do   you   think   Ian   would   come   on   the   podcast? He   might,   but   um,   he   probably   has   some   good   stories.
      </p>

        <cite>Conor</cite>
      <time>35:42</time>
      <p>
          Oh   yeah.
      </p>

        <cite>Jared Hoberock</cite>
      <time>35:43</time>
      <p>
          So   I   mean,   uh   if   my   uh   history   is   correct,   you   did   thrust,   Ian   before   CUDA   was   a   thing,   did   something   called   Brooke.
      </p>

        <cite>Conor</cite>
      <time>35:56</time>
      <p>
          Yeah,   right.   Which   was   uh   basically   Kuda   before   it   was   CUDA.
      </p>

        <cite>Jared Hoberock</cite>
      <time>36:01</time>
      <p>
          I   think   it   was   uh   quite   different   from   uh   the   KUDA.   I   think   it   had   a   much   different   programming   model   than   than   CUDA   does.   Thank   you   for   doing   this,   Jared.   Uh   thanks   for   having   you   all.
      </p>