

        <cite>Aaron Hsu</cite>
      <time>0:00</time>
      <p>
          Tercity   empowers   economical   expression,   and   if   you   have   economical   expression,   it   allows   you   to   be   simpler   longer   for   more   complexity.
      </p>

        <cite>Conor</cite>
      <time>0:09</time>
      <p>
          We&#39;ve   talked   about   before   how   algorithms   in   C   are   so   important.   There&#39;s   a   lot   of   people   that   say,   why   do   I   have   to   go   learn   these   algorithms?   A   for   loop   is   good   enough.   And   we&#39;ve   advocated   on   this   podcast   that   algorithms   elevate   your   vocabulary.   If   algorithms   are   your   vocabulary,   notation   is   the   analog   of   that   for   thought.   Cheers.   Cheers.   That   was   amazing.   Welcome   to   ADSP   the   podcast,   episode   197,   recorded   on   August   21st,   2024.   My   name   is   Connor,   and   today   I   interview   Aaron   Shu   on   site   at   the   Eagle   Pub   in   the   University   of   Cambridge.   In   today&#39;s   episode,   we   chat   about   the   importance   of   algorithms   and   tersity   in   programming   languages.
      </p>

        <cite>Aaron Hsu</cite>
      <time>1:01</time>
      <p>
          Cheers.   Cheers?
      </p>

        <cite>Conor</cite>
      <time>1:04</time>
      <p>
          Alright.   Alright.   Get   into   the   serious   stuff.   We&#39;re   back,   folks.   It&#39;s   episode,   I   didn&#39;t   even   mention   what   episode   it   was   last   time.   It&#39;s   episode,   I   think,   197   now.   We   just   got   done   recording   196.   We&#39;re   back   with   Aaron   Shu,   famous   for   his   GPU   code   defunds   compiler.   We   just   finished   talking   about   algorithms.   We   might   have   to   make   it   a   recurring   thing   every   every   few   months.   We&#39;ll   bring   you   back   to   talk   about   algorithm   implementations   on   the   CPU   versus   the   GPU.   And   uh   well,   you   can   choose   the   one   in   advance   that   you   want   to   talk   about   because   it&#39;s   uh   I   think   it   is   interesting   to   talk   about   the   trade-offs,   and   and   that&#39;s   an   interesting   thing   about   the   dialogue   implementations.   A   lot   of   time   they   have   these   kind   of   upfront   heuristic   things.   They&#39;ll   do   like   a   single   pass   reduction   to   collect   some   kind   of   like   bool   Boolean   predicate   data.   Yeah.   And   then   from   there   they&#39;ll   dispatch   to   one   of   two   or   three   algorithms.   Or   fifteen.   Well,   we   were   gonna   we&#39;re   gonna   have   to   talk   about   those.   Yeah.   And   so   it&#39;s   like   when   you   talk   like   I&#39;ve   I&#39;ve   known   that   I&#39;ve   talked   to   both   uh   Adam   and   Marshall,   who   both   Adam   still   is   working   in   dialogue   and   Marshall   used   to,   about   you   know,   what   is   the   algorithm   implementation?   They&#39;re   like,   well,   you   know,   it&#39;s   it   depends   on   your   data.   Yeah.   Anyways,   a   topic   for   another   day.   Yeah.   Yeah.   The   question   that   we&#39;re   gonna   talk   about   now   as   previewed   in   196,   episode   196.   We&#39;ll   link   it   in   the   show   notes.   Aaron   recently   talked   at   LambdaConf.   I   know   you&#39;ve   given   several   talks   in   years   past,   and   I   can&#39;t   actually   remember   which   talk   it   was,   I&#39;ll   link   both   of   them.   But   there   was   a   a   key   moment   where   you   kind   of   made   an   offhand   comment,   and   it   was   that   tersity   is   fundamental,   it&#39;s   like   underpins   the   power   of   APL.   And   without   the   tersity   of   APL,   you   don&#39;t   have   APL.   Yep.   And   this   is   something   that   I   believe,   like   inside,   like   deep   in   my   core,   but   I   arguing   that   just   like   just   because   something&#39;s   terse,   it   makes   it   powerful,   right?   Is   something   that   like   I   don&#39;t   know   how   to   pitch   that   argument.   Whether   it&#39;s   APL   or   just   like,   you   know,   having   some   language   that&#39;s   on   the   command   line,   whether   it&#39;s   you   know,   said   awk,   there&#39;s   a   lot   of   other   languages,   JQ   for   JSON,   that   aren&#39;t   necessarily   APO,   but   there&#39;s   a   power   to   grep.   There&#39;s   there&#39;s   a   power,   even   regex,   to   these   languages   that   are   the   opposite   of   verbose.   Yeah.   And   you   made   this   comment,   and   you   are   much   more   articulate   when   it   comes   to   advocating   for   the   things   that   you   believe   in.   So   this   is   my   question   is   uh   why   and   why   should   people   care   about   and   think   about   the   verbosity   of   their   code   and   what   whether   they   can   make   it   terser   to   make   it   more   powerful?
      </p>

        <cite>Aaron Hsu</cite>
      <time>3:38</time>
      <p>
          All   right,   here   we   go.   Go.   Yeah,   we   gotta   we   gotta   settle   in   for   something   like   this   because   this   is   this   is   not   an   easy   question   to   strongly   support   because   there&#39;s   so   much   institutional   momentum   around   the   concept   of   quote-unquote   readable   code   as   juxtaposed   against   tersity   there   and   this   idea   of   abstraction   and   additive   code   and   a   lot   of   assumptions   that   people   make,   assuming   uh   certain   things   about   just   the   reality   of   software   engineering,   that   would   force   you   into   the   conclusion   that   tersity   is   a   dangerous   tool,   right?   And   the   closest   we   get   in   most   arguments   is   that,   well,   you   would   like   your   code   to   be   smaller,   up   to   the   point   where   you   start   not   violating   certain   types   of   style   rules   that   are   usually   pretty   verbose.   So   I   think   the   the   first   thing   we   we   need   to   understand   is   why   is   it   so   hard   for   people   to   intuitively   embrace   the   idea   of   tersity,   right?   And   in   this   respect,   we   have   to   look   at   the   history   of   the   sort   of   the   the   wisdom   of   what   good   code   should   look   like,   right?   And   so   in   the   past,   you   have   the   60s,   the   70s,   the   wild   west   of   programming.   People   don&#39;t   really   care   too   much   about   how   your   code   looks.   What   they   care   about   is   just   solving   some   stinking   problems,   right?   Like   just   get   anything   to   work.   I   don&#39;t   care   if   you   have   to   self-rewrite   your   own   memory   to   fit   it   on   the   punch   card   to   make   the   thing   work,   right?   And   the   c   and   in   those   spaces,   cleverness   ruled   the   day,   right?   Because   you   had   to   be   really   clever   figuring   out   exactly   what   it   is   you   are   doing   on   these   very   esoteric   architectures,   each   of   which   is   bespoke.   You&#39;re   working   down   at   the   machine   level,   even   if   you&#39;re   at   a   high-level   language,   right?   But   something   happens,   right?   Something   shifts,   and   we   begin   to   scale   our   programs   up,   and   we   don&#39;t   have   tools   to   scale   software   up.   We   don&#39;t   know   the   tools,   we   don&#39;t   have   a   theory   for   how   to   scale   software.   And   this   spawns   off   the   modern   software   engineering   craze,   where   people   are   trying   to   figure   out   how   do   we   make   bigger   and   bigger   pieces   of   software,   how   do   we   make   them   run,   how   do   we   make   them   reliable,   how   do   we   stop   having   things   blow   up   in   our   faces,   sometimes   literally,   right?   And   this   whole   wave,   I   would   say   really   strongly   gets   a   lot   of   its   push   initially   from   what   we   call   imperative   programming   today,   but   which   at   the   time   was   called   structured   programming.   So   we   get   the   idea   that   you   can   use   structured   programming   to   get   a   handle   on   uh   complexity,   right?   And   and   from   the   very   beginning,   people   recognized   that   management   of   complexity   was   the   biggest   challenge   in   software.   That   people   have   understood.   And   everybody   since   then   has   understood   things   like   incremental   development,   the   need   for   breaking   things   down   into   small   pieces,   the   need   to   somehow   absorb   this   information   in   a   way   that&#39;s   accessible   to   the   human   mind,   right,   to   solve   these   problems.   But   uh   structured   programming,   imperative   programming,   comes   with   it   to   the   Dykstra   school   of   thought,   which   is   this   idea   of   formal   methods   around   programming.   The   idea   that   you   can   be   very   rigorous   about   how   you   think   about   the   structure   of   your   programs   and   sort   of   prove   important   things.   You   get   Tony   Hoare   who   develops   the   whore   logic   around   state   and   managing   memory   and   things   like   that,   and   this   spawns   a   whole   base   of   early   axiomatic   systems   on   how   to   reason   about   programs,   and   they   all   are   based   on   structured   programming,   right?   So   the   entire   ethos   of   how   to   make   reliable   code   stems   from   the   structured   programming   models,   which   then   turn   into   object-oriented   models.   All   of   this   that   you&#39;re   saying   then   doesn&#39;t   necessitate   verbosity.   Like   so   it&#39;s   it   doesn&#39;t,   well,   it   kind   of   does,   but   not   for   the   reasons   you   might   think,   right?   The   verbosity   comes   from   the   core   abstractions   that   became   foundational   at   that   time,   right?   So   the   core   abstractions   at   that   time   were   what   you   would   see   in   C.   What   eventually   became   C,   these   are   the   core   structured   programming   foundations   that   you   get   in   your   early   Fortran,   Algol,   C,   uh   Cobalt   type   programs.   You   get   these   if-then-else   clauses,   these   switch   statements,   these   loops,   uh   the   while   loop,   the   for   loop,   the   the   the   and   there   were   arguments   about   which   ones   were   better,   but   it   all   put   your   data   at   a   certain   level   of   abstraction.   And   it   was   really   the   scalar   abstraction   level,   right?   And   all   of   this   was   very   operational   and   organized   around   the   machines   at   the   time,   which   had   very   limited   register   sets   on   scalar   operations.   And   in   the   80s,   there   was   a   little   bit   of   a   parallel   computation   wave.   People   were   trying   to   come   up   with   massively   parallel   systems.   Um   and   that   ironically   was   the   time   at   which   APL   also   had   a   lot   of   uptake   and   was   one   of   the   dominant   languages.   He   said   APL.   For   a   second   I   thought   he   said   JPL,   and   I   was   like,   that&#39;s   the   NASA   just.   And   there&#39;s   so   there&#39;s   this   institution   of   software   engineering   that   latches   on   to   latches   on   to   structured   programming   and   object-oriented   programming   around   all   of   us.   And   so   there&#39;s   this   core   foundational   abstraction   that&#39;s   chosen.   And   it&#39;s   chosen,   and   if   you   look   at   the   arguments   between   that   and   APL   at   the   time,   there&#39;s   this   uh   a   lesson   begins   to   emerge   that,   well,   this   other   type   of   code   is   more   readable,   right?   And   people   start   saying,   well,   APL   is   a   write-only   language.   So   terse   languages,   the   representatives   of   terse   languages,   begin   to   get   this   reputation   of   spaghetti   code.   And   spaghetti   code   is   one   of   these   things   that   everybody&#39;s   terrified   of   at   the   time,   right?   And   nobody   really   has   theory   to   prove   any   of   this.   It&#39;s   just   who   chooses   what   to   do,   and   there&#39;s   this   giant   success   that   people   are   seeing   with   software   engineering.   And   during   the   90s,   object-oriented   programming   was   supposed   to   save   the   world   in   some   sense,   right?   And   you   get   Java,   you   get   these   garbage   collectors,   you   get   all   these   systems   that   support   this   abstraction   model.   And   all   of   that   begins   to   then   develop   into   what   does   good   code   look   like?   And   the   and   the   vast   majority   of   people   arguing   about   what   good   code   looks   like   are   already   steeped   in   this   core   foundational   abstraction   that   they   built   up.   And   in   that   foundation,   what   you&#39;re   dealing   with   is   these   modules   that   quickly   exceed   the   human   capacity   for   reasoning   around   them,   right?   So   people   start   to   then   say,   oh,   well,   it&#39;s   these   formal   methods   can&#39;t   be   handled   by   human   minds   at   scale.   This   is   one   of   the   classic   problems   that   people   have.   So   then   you   begin   to   get   this   principle,   this   proverb   in   your   head,   that   we   will   never   be   able   to   understand   the   full   system.   And   since   we   can   never   understand   the   full   system,   the   key   is   to   make   sure   that   we   break   it   apart   in   such   a   way   that   we   never   have   to   think   about   the   whole   system.   Right?   Think   small,   think   modular.   And   then   we   start   thinking   about   as   we   grow   software   teams,   we   still   have   this   sort   of   tailor-esque   attitude   towards   software   management,   which   is   very   hierarchical,   very   control-oriented,   very   much   write   this   specification   and   then   have   programmers   implement   it.   Well,   if   you&#39;re   gonna   write   a   specification   and   have   programmers   implement   it,   you   have   to   divvy   up   your   programming   work   in   such   a   way   that   each   programmer   can   really   freely,   independently   operate   on   given   pieces   of   the   software   without   ever   really   understanding   the   whole   thing,   right?   And   that   comes   from   the   management   perspective.   And   then   you   get   this   idea   that   to   make   this   clean   code,   you   have   to   design   these   abstractions   and   these   architectures   to   separate   everything   out   so   that   we   can   change   everything.   And   this   was   sort   of   a   way   forward.   And   if   you   get   to   that   point   and   you&#39;ve   sort   of   swallowed   those   ideas,   well,   you   can&#39;t   be   terse   about   that   stuff.   Because   there&#39;s   so   many   points   in   there.   You&#39;re   you&#39;re   really   you&#39;re   at   a   dangerous   point   if   you&#39;re   you&#39;re   you&#39;re   you   start   to   you   start   wanting   to   not   be   clever,   you   and   you   have   systems   that   begin   to   grow   very   quickly   because   the   inherent   complexity   of   the   problem   also   starts   inheriting   inherent   complexity   in   the   architecture.   So   your   architectures   start   to   get   big,   your   object   hierarchies   start   to   get   big,   everything   starts   to   sort   of   grow   faster.   And   that   means   that   you   you   get   these   clean   code   books,   like   Bob   Martin&#39;s   clean   code   book,   and   these   all   are   sort   of   antithetical   to   a   terse   programming   style.   And   these   are   lauded   as   the   best   way   to   do   everything.   So   everybody   seems   to   get   this   idea   that   tersity   is   a   bad   thing,   and   they   they   don&#39;t   know   why,   but   they   have   this   strong   institution   opinion   institutional   opinion   that   that&#39;s   the   way   to   go,   right?   Now,   all   of   that   is   this   big   drive   to   push   it,   and   that   means   that   it&#39;s   really   hard   to   change   an   idea   like   that   because   tersity   is   to   to   suggest   terse   code   almost   flies   in   the   face   of   all   of   that   expectation   built   up   over   decades   that   we&#39;re   gonna   work   with.   So,   in   order   to   undermine   that   idea,   you   really   have   to   go   back   to   a   question   of   how   you&#39;re   going   to   how   you&#39;re   going   to   approach   the   entire   process   of   designing   software   to   begin   with.
      </p>

        <cite>Conor</cite>
      <time>12:53</time>
      <p>
          So   going   all   the   way   back   to   impaired.   So   to   recap   too,   what   I   heard,   and   I   might   miss   a   couple   points,   but   it   you   start   with   the   rise   of   imperative   programming   back   in   the   day   called   structural   programming.   That   leads   ultimately   to   abstraction,   which   leads   to   more   abstraction,   which   leads   to   an   ability   to   not   understand   the   full   program,   which   leads   to   modularization.   And   you   didn&#39;t   say   microservices,   but   that&#39;s   what   I   was   kind   of   thinking   about.   And   at   this   point,   everything&#39;s   become   so   fractured,   you&#39;re   not   gonna   be   terse   about   it.   It&#39;s   gonna   end   up   being   verbose.   And   it   all   started   with,   you   know,   this   paradigm   of   imperative   programming   that   led   to   this   kind   of   stuff.
      </p>

        <cite>Aaron Hsu</cite>
      <time>13:35</time>
      <p>
          And   I   think   I   think   where   it   gets   the   inflection   point,   if   you   will,   is   the   idea   of   black   box   information   hiding   as   object-oriented   programming   promotes   it.   So   the   idea   that   you   create   these   objects   that   hide   state   in   a   way   that   is   mechanically   enforced   is   sort   of,   I   think,   the   inflection   point   for   this   whole   space.   And   famously,   Knuth   didn&#39;t   like   this   idea,   right?   It&#39;s   one   of   his   pet   peeves   that   everybody   sort   of   disagrees   with.   He   doesn&#39;t   like   black   box   libraries.   He   wants   editable,   reusable   code.   Right?   So   he   likes   reusable   code   that   you   can   edit.   So   he   sort   of   doesn&#39;t   want   that   information   hiding   at   the   source   level.
      </p>

        <cite>Conor</cite>
      <time>14:12</time>
      <p>
          Wait,   so   this   is   uh   across   two   episodes,   the   second   time   Knuth   has   come   up,   was   he   an   APL   fan?
      </p>

        <cite>Aaron Hsu</cite>
      <time>14:17</time>
      <p>
          No,   uh   because   what   he   liked   to   do   is   play   with   algorithms   at   the   bit   levels.   And   at   the   time,   APL   was   not   suitable   for   doing   the   kinds   of   low-level   programming   that   he   wanted.   He   thought,   he   famously,   I   think,   said   uh   famous   eh   it&#39;s   famous   to   me.   He   said   APL   is   a   terrific   problem   solving   language   and   not   the   kind   of   programming   language   he   wants   to   do,   because   he   likes   to   solve   programming   problems.   He   likes   to   play   with   CPUs   and   algorithms,   and   he   and   APL   is   the   thing   that   takes   all   of   the   algorithms   he   might   make   and   makes   them   trivially   accessible   to   the   problem   solver.   And   the   problem   solver   is   more   interested   in   solving   the   problem   than   in   worrying   about   he&#39;s   re   whether   he&#39;s   represented   this   bit   or   that   bit   in   exactly   the   precise   way   that&#39;s   gonna   map   onto   the   exact   hardware   machine.   Whereas   Knut   like   just   goes   Google   Gaga   over   that   kind   of   stuff.   Yeah,   right.   Even   inventing   his   own,   you   know,   microcode   more   or   less,   to   an   instruction   set   to   do   what   he   wants   to   do.   So   but   yes,   let&#39;s   bring   up   so   we   gotta   go   back   to   square   one.   And   we   have   to   ask   ourselves   are   there   alternatives   to   managing   complexity?   Right?   Because   complexity   is   always   gonna   be   a   problem.   We   have   to   manage   that,   right?   And   so   if   we   look   at   the   ways   that   humans   managed   complexity   in   the   past,   sometimes   uh   there   is   this   concept   of   modularization   and   other   things,   right?   But   there&#39;s   another   concept   that   actually   comes   closer   to   the   way   we   think   the   math   uh   computers   work,   and   that&#39;s   our   mathematical   history.   And   our   mathematical   history   is   the   exact   opposite.   Math   still   has   the   problem   of   complexity,   having   to   break   problems   down,   having   to   separate   concerns,   solve   issues,   but   they   also   have   the   need   to   leverage   a   lot   of   human   thought   about   really   formal   things,   right?   So   it   shifts   the   idea   from   construction   models,   which   software   engineering   sort   of   embraced   the   construction   metaphor,   to   the   the   metaphor   of   reasoning   about   concepts   or   reasoning   about   objects,   reasoning   about   ideas,   coming   up   with   properties   and   statements   and   things   like   this.   And   if   you   look   at   the   historical   development   of   formal   thought   in   humans   over   time,   when   they&#39;re   trying   to   communicate   with   one   another,   all   of   that   tends   to   evolve   towards   something   that   looks   like   math   notation.   And   one   of   the   hallmarks   that   makes   math   notation   work   is   its   tersity.   The   reason   somebody   uses   math   notation   instead   of   just   writing   out   the   expression   that   could   be   equally   formal   in   natural   language   is   that   it&#39;s   way   shorter   and   it&#39;s   much   easier   to   chunk   into   patterns   for   the   human   mind.   And   so   if   we   look   at   like   cognitive   science   research   and   we   look   at   HCI   research   and   things   like   that,   and   we   look   at   how   the   mind   engages   with   uh   information   ontologies,   this   is   something   we   have   research   on   now   that   we   didn&#39;t   have   research   on   when   this   initial   debate   was   happening.   And   we   see   that   humans   don&#39;t   do   well   with   trying   to   reason   about   abstract   frameworks   that   can&#39;t   be   visualized.   The   ability   for   your   typical   human   to   reason   about   very   complex   graphs   in   their   head   and   structures   is   really   bad.   That   you   know,   that&#39;s   where   the   common   like   seven   working   items   in   memory   kind   of   thing   comes   from,   right?   Is   there&#39;s   a   general   issue.   And   so   how   do   humans   get   over   that   hump   when   they   have   to   deal   with   massive   complexity   in   the   world?   Well,   all   of   our   mental   processing   power   uh   tends   to   go   into   a   type   of   pattern   recognition   space,   right?   A   lot   of   what   we   do   is   pattern   recognition   linking   to   various   ideas.   Uh   that&#39;s   where   our   neural   network   concepts   came   from   in   the   computer   science   world.   That&#39;s   where   case-based   reasoning   began   to   get   in   the   early   symbolic   AI   space.   Uh   that&#39;s   where   we   recognize   the   idea   of   chunking   in   in   memory   research,   where   you   humans,   in   order   to   uh   think   about   complex   topics,   they   will   chunk   more   complex   things   into   units   that   can   be   stored   and   referenced   as   a   single   atomic   entity.   And   it   helps   to   build   these   graphs   in   your   head   that   you   can   think   of   at   different   abstraction   layers   through   atomic   units   and   these   chunks.
      </p>

        <cite>Conor</cite>
      <time>18:26</time>
      <p>
          This   sounds   similar   to   the   trick   of   trying   to   memorize   some   series   of   digits   into   like   if   you   split   it   a   nine-digit   sequence   into   three   three-digit   numbers   and   memorize   123   versus   a   couple   other   numbers.   That&#39;s   an   easier   way   to   memorize   instead   of   just   nine   single   digits.
      </p>

        <cite>Aaron Hsu</cite>
      <time>18:41</time>
      <p>
          Or   what   you   can   do   with   numbers   is   you   can   translate   groups   of   numbers   into   words,   and   then   translate   those   words   into   a   story,   and   that&#39;s   leveraging   more   processing   power   than   the   raw   memorization   skills.   So   it&#39;s   leveraging   more   of   your   brain,   makes   it   easier   to   remember,   and   it   makes   it   easier   to   recall.   And   here&#39;s   something   that   um   I   really   uh   that   that   people   have   forgotten   about   is   when   it&#39;s   easier   to   memorize   in   your   head,   it&#39;s   easier   to   immediately   reason   about   over   time   because   the   connections   that   you&#39;re   the   it&#39;s   sort   of   like   caching,   right?   If   you   have   to   go   out   to   disk   every   time   you   need   to   add   two   registers   together,   you&#39;re   really   slow.   But   if   those   registers   are   sitting   right   there   and   you   can   just   add   two   registers   together,   that&#39;s   blazingly   fast.   So   the   human   mind   has   the   same   issue,   right?   These   are   levels   of   cache.   And   most   people   in   programming   right   now   have   accepted   this   idea   that   there&#39;s   not   enough   processing   power   in   their   minds,   and   so   therefore,   we   just   have   to   store   everything   on   disk.
      </p>

        <cite>Conor</cite>
      <time>19:37</time>
      <p>
          Right,   and   constantly   reading   docs   and   it&#39;s   a   good   thing.
      </p>

        <cite>Aaron Hsu</cite>
      <time>19:39</time>
      <p>
          And   so   what   they&#39;re   doing   is   they&#39;re   essentially   constantly   invalidating   cache   in   their   head.   So   they&#39;re   not   leveraging   the   caches   that   exist   in   their   heads.   And   this   is   where   deep   learning   comes   in.   Whereas   if   you   have   something   that   you&#39;ve   memorized   in   your   head   and   you   can   connect   that   pattern   to   a   lot   of   other   patterns,   you   now   have   the   ability   to   reason   with   that   pattern   across   a   lot   of   spaces   way   faster   than   if   you   took   the   equivalent   concept   and   had   to   remember   it   by   looking   it   up   somewhere   in   an   API   reference   or   something   else.   Now,   how   do   you   apply   that   to   programming?   Well,   if   we   look   at   how   it&#39;s   applied   in   math,   you&#39;ll   notice   that   there&#39;s   a   lot   of   core   abstractions   that   mathematicians   repeatedly   use   over   and   over   and   over   across   many,   many   different   domains.   Sets.   Maybe   if   you&#39;re   a   homotopy   type   there,   you   use   types.   Uh   natural   numbers,   reels,   all   these   things,   these   concepts,   all   connect   to   each   other   and   they   form   the   basis   of   a   lot   of   mathematical   thought.   And   so   if   you   want   to   reason   about   some   other   thing,   you   encode   it   into   sets,   symbols,   numbers,   you   know,   these   other   things   that   you   now   have   this   huge   toolbox   memorized   in   your   head,   ready   to   go.   And   if   you   look   at   mathematicians,   they   have   all   these   theorems,   they   have   all   these   practices,   all   these   manipulations   that   they   can   do   in   their   heads,   and   they&#39;re   ready   to   go.   Right?   And   it&#39;s   and   that&#39;s   how   they   think   and   reason   about   programs,   and   that&#39;s   how   they   solve   the   complexity   problem.   Because   they   translate   an   idea   into   a   representation   that   they   have   this   massive   toolbox   of   things   that   are   easy   for   them   to   memorize   that   they   can   use.   But   if   you   took   a   bunch   of   these   theorems   and   you   didn&#39;t   have   the   formulas   ready   to   go,   it&#39;s   much   harder   to   apply   them.   That&#39;s   why   a   lot   of   mathematicians   use   the   formulas   or   derive   the   formulas   before   they   apply   the   theorem.   They   might   know   the   theorem,   but   then   they&#39;re   going   to   derive   the   formula   and   then   go   apply   it   because   that&#39;s   what   is   easier   to   work   with.   And   so   if   we   translate   this   to   computer   science,   if   we   want   to   achieve   the   same   type   of   reasoning   power,   expand   that   reasoning   power   in,   that&#39;s   the   notational   reasoning   power   that   Iverson   was   talking   about.   That   appeal   to   a   symbolic   language   that   allows   us   to   express   ideas   concisely   and   then   leverage   the   natural   pattern   matching,   symbolic   reasoning   that   we   have   in   our   heads   to   connect   those   ideas   with   lots   of   other   ideas   because   they   share   a   common   domain   and   they   share   a   common   symbolic   language   over   which   we   can   manipulate   those   ideas.   And   so   we   do   that   in   computer   science.   The   only   way   to   do   that   is   with   terse   languages.   The   more   verbose   your   language   gets,   the   harder   it   is   for   your   mind   to   see   the   patterns,   connect   the   patterns,   and   leverage   them   internally   to   think   about   problems.   So   it&#39;s   like   you&#39;re   constantly   invalidating   your   cache   because   what   that   verbose   code   is   too   big   to   naturally   fit   within   the   tightest   cache   regions   of   your   mind.   Whereas   a   terse   language   much   more   naturally   connects   in   with   all   of   the   caches   in   your   mind   and   allows   you   to   reason   more   easily   about   those   problems.   Especially   over   time.   So   in   APL,   this   is   the   array   abstraction.   In   Lisp,   this   is   the   linked   list   cons   model   of   programming.   Both   of   them,   if   you   can   translate   your   ideas   into   those   data   representations,   now   you&#39;ve   got   this   giant   suite   of   tools   that   you   can   apply   to   that   data   representation   that   you   already   know   how   to   use,   and   you   can   begin   to   see   connections   between   ideas.   What   it   does   is   it   elevates,   it   elevates   the   base   layer   of   complexity   that   an   individual   can   manage.   And   so   it   doesn&#39;t   invalidate   all   of   the   lessons   learned   in   some   of   the   software   engineering   stuff.   But   what   it   does   is   it   shifts   the   scope.   So   the   scope   of   what   an   individual   person   or   mind   can   take   on   suddenly   becomes   much   larger   problem   domains.   Right?   And   so   now   we   don&#39;t   need   the   overhead   of   the   software   engineering   abstractions   for   a   huge   number   of   programs   because   they&#39;re   too   trivial   to   need   that   anymore.   Because   our   human   minds   can   handle   it   now.   So   by   using   tersity   to   expand   the   range   of   problems   that   we   can   tackle   as   single   people,   we   expand   the   range   that   we   can   tackle   as   a   whole.   And   we   tackle   the   complexity   problem   basically   by   by   removing   the   uh   making   it   easier   for   the   human   to   tackle   that   complexity   directly,   using   the   human   mind.   That&#39;s   where   the   tools   of   thought   come   into   play.   So   then   we   still   have   to   do   software   engineering   once   we   exceed   that   level,   right?   There&#39;s   still,   when   two   people   have   to   share   an   idea   over   space   and   time   and   it   begins   to   exceed   the   capacity   for   the   trivial,   we   now   have   to   add   a   little   bit   more   to   manage   it.   But   we&#39;re   talking   about   the   day,   like   to   use   my   compiler   as   an   example.   Most   compilers   have   obscene   amounts   of   abstraction   in   them   because   they   need   them   to   manage   these   ideas   over   the   base   abstraction   layer   that   they&#39;re   working   with.   But   since   I&#39;m   writing   my   compiler   using   APL,   I   don&#39;t   need   most   of   those   abstractions.   So   all   I   need   to   do   is   occasionally   create   a   function   uh   function   abstraction   for   what   we   would   normally   think   of   as   a   large   module   or   multiple   submodules   of   an   idea   that&#39;s   linking   against   other   concepts.   And   I   can   collapse   all   of   that   down   to   the   idea   of   a   function   abstraction.   That&#39;s   a   much   simpler   concept   than   a   visitor   pattern   of   a   particular   class   with   a   particular   subclass   derived   from   this   particular   sub-AST   structure   that&#39;s   going   to   then   be   applied   using   these   loops   and   these   for   loops   and   this   recursion.   All   of   those   concepts   are   necessary   to   do   the   similar   thing   for   a   single   pass.   Whereas   for   me,   I   have   a   single   function   that   might   contain   20   compiler   passes,   and   each   of   those   compiler   passes   needs   no   additional   abstraction,   and   it&#39;s   a   few   lines   of   code.   And   so   it&#39;s   in   from   an   informational   theoretical   standpoint,   it&#39;s   much   more   efficient,   and   therefore   it&#39;s   a   lot   easier   for   our   minds   to   engage   with.   Because   it   maps   patterns   that   are   much   easier   to   reuse   efficiently,   quickly,   internally.   And   so   that   still   means   we   have   to   be   aware   of   what   software   engineering   has   given   us.   You   know,   these   ideas   of   arranging   your   data   in   such   a   way   that   you&#39;ve   got   independence.   Doing   all   this,   but   we   can   often   do   it   without   needing   the   extra   overheads   that   are   traditionally   needed   if   we   embrace   tersity.   Now   we   have   to   develop   skills   around   how   we   use   tersity   effectively.   We   have   to   understand   what   we&#39;re   gaining   from   using   tersity   and   apply   it   in   that   fashion.   Just   like   we   have   you   cannot   over-abstract   with   object   hires   and   generics   and   type   classes   or   whatever   in   your   OOP   language.   If   you   do   that,   you   end   up   with   spaghetti   code   just   like   anybody   else,   and   it&#39;s   horrible   code.   You   have   to   learn   how   to   use   your   tools   effectively.   And   tersity   is   another   one   of   those   tools   you   need   to   learn   to   use   effectively.   But   it   allows   you   to   obviate   the   need   for   a   whole   lot   of   other   tools   by   just   leveraging   one   simpler   tool   if   you   learn   how   to   use   it.
      </p>

        <cite>Conor</cite>
      <time>26:47</time>
      <p>
          I   mean,   take   a   bow.   That   was   I&#39;ve   I&#39;ve   been   sitting   here   like   deep   in   thought   while   listening   to   you.   Cheers.   Cheers.   That   was   amazing.   We&#39;re   at   a   table   that   has   a   reservation   in   eight   minutes,   so   we&#39;ll   we&#39;ll   wrap   this   up   and   uh   land   this   plane   before   them.   But   that   every   once   in   a   while   you   have   a   conversation   where   it   like   lights   up   your   brain   because   almost   everything   you   were   saying   there   made   me   think   about.   I&#39;ll   have   to   iterate   on   this   quote   that   I   have   in   my   head,   but   there&#39;s   something   like   on   ADSP,   we&#39;ve   talked   about   before   how   algorithms   in   C   are   so   important.   There&#39;s   a   lot   of   people   that   say,   Why   do   I   have   to   go   learn   these   algorithms?   A   for   loop   is   good   enough.   And   we&#39;ve   advocated   on   this   podcast   that   algorithms   elevate   your   vocabulary.   So   and   there&#39;s   an   analogy   here   between   everything   you   were   saying,   and   in   the   back   of   my   head,   I&#39;m   like,   it&#39;s   it&#39;s   making   me   think   that   what   algorithms   in   C   do   for   your   vocabulary,   tercity   does   for   your   ability   to   comprehend   and   solve   problems.   Right.   Right.   So   like   the   same   way   that   a   vocabulary   of   algorithms   like   rotate   and   reverse   and   reduce   and   scan,   that   vocabulary   makes   it   easier   for   you   to   communicate   with   your   coworkers   about   soft,   like   if   I   have   to   say   a   for   loop   that   has   an   if   thing   that   checks,   no,   it&#39;s   a   find   if,   it&#39;s   a   reduction.   The   same   thing   that   that   vocabulary   does   for   you   when   communicating   verbally   and   even   in   code   with   your   coworkers,   the   same   thing   tersity   does   for   you   when   it   comes   to   architecture   and   solving   problems   and   your   ability   to   look   at   a   screen   of   code   and   see   the   whole   solution,   even   if   there&#39;s   a   lot   of   stuff   going   on,   you   can   see   it   on   a   single   screen   or   in   a   single   function,   whereas   in   an   imperative   paradigm,   you&#39;re   gonna   need   all   this   abstract   you   might   not   even   in   the   imperative   style   need   the   abstraction,   but   that&#39;s   just   the   way   that   idiomatically   it&#39;s   done.   You&#39;re   using   classes   and   stuff.   There&#39;s   some   analogy   there   that   uh   it&#39;s   really   making   me   think   that   we   we   had   a   two-hour   discussion   the   other   day   about   whether   libraries   are   necessary.   And   uh   I   I   wasn&#39;t   fully   convinced   or   at   all   convinced   that   we   don&#39;t   need   them.   But   everything   you   said   over   the   last   15   minutes   or   so,   I   can   see   how   you   get   to   the   uh   a   less   of   a   need   for   libraries.   If   algorithms   are   your   vocabulary,   notation   is   the   analog   of   that   for   thought   or   something   like   that.
      </p>

        <cite>Aaron Hsu</cite>
      <time>29:10</time>
      <p>
          The   the   vocabulary   and   the   uh   tersity   empower   each   other.   So   if   you   have   a   well-selected   vocabulary   that   is   expressive   and   general,   then   uh   if   it&#39;s   an   economical   vocabulary,   then   a   terse   notation   for   using   those   vocabulary   terms   allows   you   to   chain   them   together   in   patterns   that   allow   you   to   use   them   more   effectively,   more   easily,   and   scales   their   applicability   farther.
      </p>

        <cite>Conor</cite>
      <time>29:39</time>
      <p>
          Right,   right.   Right?   So   And   then   you   use   the   word   elevate   at   one   point,   and   I   was   literally   thinking   that   like   you&#39;re   elevating   not   just   your   ability   to   think   in   your   head,   but   your   ability   to   iterate   and   solve   because   it&#39;s   one   of   my   biggest   complaints   when   swapping   between   APL   and   C   is   I   can   try   out,   I   literally   have   a   talk   called   Algorithms   as   a   Tool   of   Thought   where   I   show   10   or   11   different   talk.   You   provided   solution   number   nine   or   something,   and   it&#39;s   like   10   different   solutions   in   30   minutes   in   a   slide   deck.   All   of   them,   I   think   the   most   was   six   characters,   and   most   of   them   are   three   or   four   or   five.   In   C   you   can&#39;t   give,   I   mean,   you   probably   could,   but   you&#39;re   burning   through   those   solutions   because   every   single   one   of   those   primitives   in   APL   is   the   hopefully   the   equivalent   of   an   algorithm   in   C   ⁇ ,   but   not   all   of   them,   so   you&#39;re   gonna   end   up   spelling   out   some   boilerplate,   and   like   the   making   of   that   talk   in   APL,   it&#39;s   like   an   order   of   magnitude,   if   not   less,   than   the   equivalent   preparation   for   C   code   for   because   coding   the   equivalents,   they   don&#39;t   take,   they   don&#39;t   just   like   the   namespaces   on   a   single   operation   in   the   C   code   already   exceeds   the   length   of   the   solution   in   AP.
      </p>

        <cite>Aaron Hsu</cite>
      <time>30:45</time>
      <p>
          And   just   the   difficulty   of   presenting   that   kind   of   code   to   an   audience   is   hugely   magnified.
      </p>

        <cite>Conor</cite>
      <time>30:52</time>
      <p>
          Yeah,   yeah.   And   and   there&#39;s   so   much   more   just   the   namespaces,   like   when   it   comes   to   C,   I   am   definitely   an   algorithm   expert.   And   there&#39;s   like   four   different   now,   you   need   to   know   the   the   iterator   algorithms   from   C   98,   plus   the   ones   that   were   added   in   14,   17.   Then   in   20   you   get   the   ranges   overloads   of   them.   Then   we   get   views,   which   technically   are   not   algorithms,   but   you   call   them   it   anyway.   So   there&#39;s   like   four   or   five   different   things.   Yeah.   And   then   there&#39;s   the   parallel   algorithms   that   I   didn&#39;t   even   mention.   It   is   impossible   for   a   beginner   to   keep   that   stuff   in   their   head.   I&#39;ve   been   doing   this   since   like   2014.   Yeah.   Whereas   in   APL,   it&#39;s,   you   know,   it   was   designed   well   from   the   get-go.
      </p>

        <cite>Aaron Hsu</cite>
      <time>31:25</time>
      <p>
          This   is   a   question   of   economy,   right?   Is   tersity   empowers   economical   expression,   and   if   you   have   economical   expression,   it   allows   you   to   be   simpler   longer   for   more   complexity,   right?   You   the   you   can   stay   simple   for   more   complex   problems.   If   you   have   an   the   ability   to   be   economical   with   your   vocabulary,   with   your   domain,   with   your   expression,   your   notation,   everything.   Whereas   as   things   become   more   difficult,   you   need   more   to   get   a   handle   on   it.   And   as   you   get   more,   you   need   more.   And   C   is   the   poster   child   of   needing   more   to   do   more.   And   it   like   they   they   make   the   right   choice   for   what   they   need   to   do   because   of   the   constraints   they&#39;re   working   under,   but   it   leads   to   a   proliferation   of   vocabulary   and   syntax   and   names   and   everything.   Whereas   though   that&#39;s   all   fundamentally   unnecessary,   even   if   it&#39;s   practically   required   for   them   as   a   design   problem   at   the   space   that   they&#39;re   at   right   now.
      </p>

        <cite>Conor</cite>
      <time>32:24</time>
      <p>
          Is   it   possible   for   a   non-symbolic   language   to   gain   the   power   of   tersity?
      </p>

        <cite>Aaron Hsu</cite>
      <time>32:31</time>
      <p>
          It   is   it   is   possible,   but   it   requires   an   intentional   rejection   of   the   tools   that   the   languages   provide   you,   which   is   very   difficult   because   you   can&#39;t   use   all   those   syntactic   extras.   You   have   to   strip   it   down   and   simplify   your   expressions   so   that   you   can   try   to   take   advantage   of   at   least   a   little   bit   of   tersity,   right?   You   don&#39;t   have   you   can   get   a   lot   of   advantages   without   going   all   the   way   to   the   symbols,   but   there   isn&#39;t   a   really   strong   inflection   point   that   you   hit   really   quickly   if   you   don&#39;t   leverage   symbols   to   the   full   effect.   Because   there&#39;s   a   big   difference   between   a   page   of   code,   a   line   of   code,   and   you   know,   a   few   pair,   you   know,   a   couple   of   pages   of   code.   And   every   time   you   double   or   triple   your   size,   you   you   are   cutting   out   a   massive   amount   of   work   that   you   can   do,   right?   So,   you   know,   in   my   compiler,   four   lines   of   code   is   a   huge   difference   between   that   and   like   say   20   lines   of   code.   So   you   can   write   some   really   nice,   small,   clean   C   code,   but   you&#39;re   st   you&#39;re   only   going   part   way   there   and   you&#39;re   still   gonna   hit   really   hard   barriers   until   you   sort   of   swallow   the   entire   concept.   You   know,   and   some   people   just   are   not   gonna   want   to   swallow   that   concept,   but   that&#39;s   where   the   biggest   power   lies.   And   that&#39;s   why   math   notation   inevitably   needs   its   symbols.   You   can&#39;t   really   do   the   math   without   the   symbols.   People   would   just   invent   more   symbols   at   some   point.   Because   once   you   get   to   a   certain   point   of   doing   real   work,   the   symbols   are   required   to   achieve   the   control   over   complexity   that   you   want.
      </p>

        <cite>Conor</cite>
      <time>34:02</time>
      <p>
          Well,   this   has   been   amazing.   I&#39;m   gonna   have   fun   editing   this   because   I&#39;m   gonna   be   deep   in   thought   again   while   I   edit   this.   Thanks   for   letting   me   rant.   This   was   this   was   the   goal   and   it   was   achieved.   Uh   thank   you   so   much.   We&#39;ll   do   this   again   sometime.   Absolutely.   Be   sure   to   check   these   show   notes   either   in   your   podcast   app   or   at   adsphepodcast.com   for   links   to   anything   we   mentioned   in   today&#39;s   episode,   as   well   as   a   link   to   a   GitHub   discussion   where   you   can   leave   thoughts,   comments,   and   questions.   Thanks   for   listening.   We   hope   you   enjoyed   and   have   a   great   day.   Low   quality,   high   quality,   that   is   the   tagline   of   our   podcast.   It&#39;s   not   the   tagline.   Our   tagline   is   chaos   with   sprinkles   of   information.
      </p>