gaspersic/triton
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[GH-PAGES] Updated website

Browse Source
This commit is contained in:
Philippe Tillet
2021-08-02 00:13:11 +00:00
parent 462b4d9bc9
commit b01d514815
20 changed files with 107 additions and 98 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
_downloads/662999063954282841dc90b8945f85ce/tutorials_jupyter.zip
View File

Binary file not shown.

BIN
_downloads/763344228ae6bc253ed1a6cf586aa30d/tutorials_python.zip
View File

Binary file not shown.

BIN
_images/sphx_glr_01-vector-add_001.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 23 KiB

After

Width:  |  Height:  |  Size: 24 KiB

BIN
_images/sphx_glr_01-vector-add_thumb.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 15 KiB

After

Width:  |  Height:  |  Size: 15 KiB

BIN
_images/sphx_glr_02-fused-softmax_001.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 39 KiB

After

Width:  |  Height:  |  Size: 39 KiB

BIN
_images/sphx_glr_02-fused-softmax_thumb.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 24 KiB

After

Width:  |  Height:  |  Size: 24 KiB

BIN
_images/sphx_glr_03-matrix-multiplication_001.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 54 KiB

After

Width:  |  Height:  |  Size: 54 KiB

BIN
_images/sphx_glr_03-matrix-multiplication_thumb.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 32 KiB

After

Width:  |  Height:  |  Size: 31 KiB

6
_sources/getting-started/tutorials/01-vector-add.rst.txt
View File

@@ -216,13 +216,13 @@ We can now run the decorated function above. Pass `print_data=True` to see the p
vector-add-performance:
size Triton Torch
0 4096.0 9.540372 9.600000
0 4096.0 9.600000 9.600000
1 8192.0 19.200000 19.200000
2 16384.0 38.400001 38.400001
3 32768.0 76.800002 76.800002
4 65536.0 127.999995 127.999995
5 131072.0 219.428568 219.428568
6 262144.0 341.333321 341.333321
6 262144.0 341.333321 384.000001
7 524288.0 472.615390 472.615390
8 1048576.0 614.400016 614.400016
9 2097152.0 722.823517 722.823517
@@ -239,7 +239,7 @@ We can now run the decorated function above. Pass `print_data=True` to see the p
.. rst-class:: sphx-glr-timing
**Total running time of the script:** ( 0 minutes 11.067 seconds)
**Total running time of the script:** ( 0 minutes 11.032 seconds)
.. _sphx_glr_download_getting-started_tutorials_01-vector-add.py:

12
_sources/getting-started/tutorials/02-fused-softmax.rst.txt
View File

@@ -262,15 +262,15 @@ We will then compare its performance against (1) :code:`torch.softmax` and (2) t
softmax-performance:
N Triton Torch (native) Torch (jit)
0 256.0 512.000001 546.133347 273.066674
1 384.0 585.142862 585.142862 261.446801
1 384.0 585.142862 585.142862 267.130429
2 512.0 630.153853 606.814814 264.258068
3 640.0 682.666684 640.000002 265.974036
3 640.0 682.666684 640.000002 269.473696
4 768.0 702.171410 664.216187 273.066663
.. ... ... ... ...
93 12160.0 812.359066 405.755985 329.483481
94 12288.0 812.429770 415.222812 329.602681
93 12160.0 812.359066 406.179533 329.483481
94 12288.0 812.429770 415.661740 329.602681
95 12416.0 810.840807 412.149375 329.173158
96 12544.0 810.925276 412.971190 329.022957
96 12544.0 810.925276 412.546756 329.292871
97 12672.0 811.007961 412.097543 329.410251
[98 rows x 4 columns]
@@ -290,7 +290,7 @@ In the above plot, we can see that:
.. rst-class:: sphx-glr-timing
**Total running time of the script:** ( 1 minutes 8.169 seconds)
**Total running time of the script:** ( 1 minutes 8.174 seconds)
.. _sphx_glr_download_getting-started_tutorials_02-fused-softmax.py:

54
_sources/getting-started/tutorials/03-matrix-multiplication.rst.txt
View File

@@ -371,37 +371,37 @@ We can now compare the performance of our kernel against that of cuBLAS. Here we
matmul-performance:
M cuBLAS ... Triton Triton (+ LeakyReLU)
0 128.0 0.455111 ... 0.512000 0.512000
1 256.0 2.730667 ... 2.978909 2.978909
2 384.0 7.372800 ... 8.507077 8.507077
3 512.0 14.563555 ... 15.420235 15.420235
4 640.0 22.260869 ... 23.272727 23.272727
1 256.0 2.978909 ... 2.978909 2.978909
2 384.0 7.372800 ... 7.899428 7.899428
3 512.0 14.563555 ... 16.384000 15.420235
4 640.0 22.260869 ... 24.380953 24.380953
5 768.0 32.768000 ... 34.028308 34.028308
6 896.0 39.025776 ... 39.025776 39.025776
7 1024.0 49.932191 ... 52.428801 52.428801
7 1024.0 51.150050 ... 52.428801 52.428801
8 1152.0 44.566925 ... 46.656000 46.656000
9 1280.0 51.200001 ... 56.109587 56.109587
10 1408.0 64.138541 ... 65.684049 65.684049
11 1536.0 80.430545 ... 76.106321 76.106321
12 1664.0 63.372618 ... 61.636381 61.636381
13 1792.0 72.983276 ... 69.379162 68.953520
14 1920.0 68.098521 ... 69.818184 69.818184
15 2048.0 73.584279 ... 75.234154 75.573044
16 2176.0 83.500614 ... 80.817862 80.494588
17 2304.0 68.251065 ... 73.051599 73.051599
18 2432.0 71.305746 ... 81.197876 81.197876
19 2560.0 77.833728 ... 76.740048 76.382283
20 2688.0 83.004501 ... 83.369354 85.051697
21 2816.0 81.067298 ... 75.982940 78.868366
22 2944.0 79.483304 ... 79.865439 79.230573
23 3072.0 81.589488 ... 83.146995 83.025078
24 3200.0 84.432717 ... 86.956520 89.385477
25 3328.0 84.003845 ... 82.843841 85.806075
26 3456.0 82.350937 ... 84.686523 84.508982
27 3584.0 84.111686 ... 92.032132 96.475743
28 3712.0 86.416391 ... 84.874549 88.170647
29 3840.0 85.005380 ... 87.771425 87.493673
30 3968.0 92.512459 ... 80.703662 84.797731
31 4096.0 93.727466 ... 90.995066 91.741443
11 1536.0 80.430545 ... 76.106321 75.296679
12 1664.0 63.372618 ... 62.061463 61.636381
13 1792.0 72.983276 ... 68.953520 68.533074
14 1920.0 69.120002 ... 68.435645 68.435645
15 2048.0 73.908442 ... 75.573044 75.234154
16 2176.0 83.500614 ... 80.173899 79.855747
17 2304.0 68.446623 ... 73.051599 72.607513
18 2432.0 71.125224 ... 81.197876 80.963875
19 2560.0 77.649287 ... 76.027843 76.740048
20 2688.0 83.552988 ... 83.186525 82.823267
21 2816.0 84.035084 ... 76.921000 79.733474
22 2944.0 82.102191 ... 80.122235 78.729910
23 3072.0 82.540970 ... 82.661468 82.661468
24 3200.0 84.432717 ... 89.385477 84.432717
25 3328.0 83.905938 ... 86.113988 86.528001
26 3456.0 82.015834 ... 83.545665 84.156124
27 3584.0 87.466332 ... 92.600816 84.988707
28 3712.0 85.163978 ... 82.902362 83.666116
29 3840.0 84.292684 ... 84.550462 85.070769
30 3968.0 89.921841 ... 87.472354 87.409694
31 4096.0 93.792965 ... 89.478485 90.260743
[32 rows x 5 columns]
@@ -411,7 +411,7 @@ We can now compare the performance of our kernel against that of cuBLAS. Here we
.. rst-class:: sphx-glr-timing
**Total running time of the script:** ( 1 minutes 58.805 seconds)
**Total running time of the script:** ( 2 minutes 2.376 seconds)
.. _sphx_glr_download_getting-started_tutorials_03-matrix-multiplication.py:

8
_sources/getting-started/tutorials/sg_execution_times.rst.txt
View File

@@ -5,12 +5,12 @@
Computation times
=================
**03:18.042** total execution time for **getting-started_tutorials** files:
**03:21.583** total execution time for **getting-started_tutorials** files:
+---------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_getting-started_tutorials_03-matrix-multiplication.py` (``03-matrix-multiplication.py``) | 01:58.805 | 0.0 MB |
| :ref:`sphx_glr_getting-started_tutorials_03-matrix-multiplication.py` (``03-matrix-multiplication.py``) | 02:02.376 | 0.0 MB |
+---------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_getting-started_tutorials_02-fused-softmax.py` (``02-fused-softmax.py``) | 01:08.169 | 0.0 MB |
| :ref:`sphx_glr_getting-started_tutorials_02-fused-softmax.py` (``02-fused-softmax.py``) | 01:08.174 | 0.0 MB |
+---------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_getting-started_tutorials_01-vector-add.py` (``01-vector-add.py``) | 00:11.067 | 0.0 MB |
| :ref:`sphx_glr_getting-started_tutorials_01-vector-add.py` (``01-vector-add.py``) | 00:11.032 | 0.0 MB |
+---------------------------------------------------------------------------------------------------------+-----------+--------+

11
_sources/programming-guide/chapter-2/related-work.rst.txt
View File

@@ -2,7 +2,7 @@
Related Work
==============
At first sight, Triton may seem like just yet another DSL for DNNs. The purpose of this section is to contextualize Triton and highlights its differences with the two leading approaches in this domain: polyhedral compilation and scheduling languages.
At first sight, Triton may seem like just yet another DSL for DNNs. The purpose of this section is to contextualize Triton and highlight its differences with the two leading approaches in this domain: polyhedral compilation and scheduling languages.
-----------------------
Polyhedral Compilation
@@ -121,7 +121,7 @@ Limitations
Unfortunately, polyhedral compilers suffer from two major limitations that have prevented its adoption as a universal method for code generation in neural networks.
First, the set of possible program transformations $\Omega = \{ \Theta_S ~|~ S \in \text{program} \}$ is large, and grows with the number of statements in the program as well as with the size of their iteration domain. Verifying the legality of each transformation can also require the resolution of complex integer linear programs, making polyhedral compilation very computationally expensive. To make matters worse, hardware properties (e.g., cache size, number of SMs) and contextual characteristics (e.g., input tensor shapes) also have to be taken into account by this framework, leading to expensive auto-tuning procedures [SATO2019]_.
First, the set of possible program transformations :math:`\Omega = \{ \Theta_S ~|~ S \in \text{program} \}` is large, and grows with the number of statements in the program as well as with the size of their iteration domain. Verifying the legality of each transformation can also require the resolution of complex integer linear programs, making polyhedral compilation very computationally expensive. To make matters worse, hardware properties (e.g., cache size, number of SMs) and contextual characteristics (e.g., input tensor shapes) also have to be taken into account by this framework, leading to expensive auto-tuning procedures [SATO2019]_.
Second, the polyhedral framework is not very generally applicable; SCoPs are relatively common [GIRBAL2006]_ but require loop bounds and array subscripts to be affine functions of loop indices, which typically only occurs in regular, dense computations. For this reason, this framework still has to be successfully applied to sparse -- or even structured-sparse -- neural networks, whose importance has been rapidly rising over the past few years.
@@ -131,7 +131,7 @@ On the other hand, blocked program representations advocated by this dissertatio
Scheduling Languages
-----------------------
Separation of concerns \cite{dijkstra82} is a well-known design principle in computer science: programs should be decomposed into modular layers of abstraction that separate the semantics of their algorithms from the details of their implementation. Systems like Halide and TVM push this philosophy one step further, and enforce this separation at the grammatical level through the use of a **scheduling language**. The benefits of this methodology are particularly visible in the case of matrix multiplication, where, as one can see below, the definition of the algorithm (Line 1-7) is completely disjoint from its implementation (Line 8-16), meaning that both can be maintained, optimized and distributed independently.
Separation of concerns [DIJKSTRA82]_ is a well-known design principle in computer science: programs should be decomposed into modular layers of abstraction that separate the semantics of their algorithms from the details of their implementation. Systems like Halide and TVM push this philosophy one step further, and enforce this separation at the grammatical level through the use of a **scheduling language**. The benefits of this methodology are particularly visible in the case of matrix multiplication, where, as one can see below, the definition of the algorithm (Line 1-7) is completely disjoint from its implementation (Line 8-16), meaning that both can be maintained, optimized and distributed independently.
.. code-block:: python
:linenos:
@@ -168,7 +168,7 @@ Scheduling languages are, without a doubt, one of the most popular approaches fo
Limitations
++++++++++++
This ease-of-development comes at a cost. First of all, existing systems that follow this paradigm tend to be noticeably slower than Triton on modern hardware when applicable (e.g., V100/A100 tensor cores w/ equal tile sizes). I do believe that this is not a fundamental issue of scheduling languages -- in the sense that it could probably be solved with more efforts -- but it could mean that these systems are harder to engineer. More importantly, existing scheduling languages generate loops whose bounds and increments cannot depend on surrounding loop indice without at least imposing severe constraints on possible schedules -- if not breaking the system entirely. This is problematic for sparse com-putations, whose iteration spaces may be irregular.
This ease-of-development comes at a cost. First of all, existing systems that follow this paradigm tend to be noticeably slower than Triton on modern hardware when applicable (e.g., V100/A100 tensor cores w/ equal tile sizes). I do believe that this is not a fundamental issue of scheduling languages -- in the sense that it could probably be solved with more efforts -- but it could mean that these systems are harder to engineer. More importantly, existing scheduling languages generate loops whose bounds and increments cannot depend on surrounding loop indice without at least imposing severe constraints on possible schedules -- if not breaking the system entirely. This is problematic for sparse computations, whose iteration spaces may be irregular.
.. table::
:widths: 50 50
@@ -206,4 +206,5 @@ References
.. [GROSSER2012] T. Grosser et al., "Polly - Performing Polyhedral Optimizations on a Low-Level Intermediate Representation", Parallel Processing Letters 2012
.. [SATO2019] Y. Sato et al., "An Autotuning Framework for Scalable Execution of Tiled Code via Iterative Polyhedral Compilation", TACO 2019
.. [GIRBAL2006] S. Girbal et al., "Semi-Automatic Composition of Loop Transformations for Deep Parallelism and Memory Hierarchies", International Journal of Parallel Programming 2006
.. [MULLAPUDI2016] R. Mullapudi et al., "Automatically scheduling halide image processing pipelines", TOG 2016
.. [DIJKSTRA82] E. W. Dijkstra et al., "On the role of scientific thought", Selected writings on computing: a personal perspective 1982
.. [MULLAPUDI2016] R. Mullapudi et al., "Automatically scheduling halide image processing pipelines", TOG 2016

6
getting-started/tutorials/01-vector-add.html
View File

@@ -305,13 +305,13 @@ for different problem sizes.</p>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>vector-add-performance:
size Triton Torch
0 4096.0 9.540372 9.600000
0 4096.0 9.600000 9.600000
1 8192.0 19.200000 19.200000
2 16384.0 38.400001 38.400001
3 32768.0 76.800002 76.800002
4 65536.0 127.999995 127.999995
5 131072.0 219.428568 219.428568
6 262144.0 341.333321 341.333321
6 262144.0 341.333321 384.000001
7 524288.0 472.615390 472.615390
8 1048576.0 614.400016 614.400016
9 2097152.0 722.823517 722.823517
@@ -323,7 +323,7 @@ for different problem sizes.</p>
15 134217728.0 851.577704 850.656574
</pre></div>
</div>
<p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 0 minutes 11.067 seconds)</p>
<p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 0 minutes 11.032 seconds)</p>
<div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-getting-started-tutorials-01-vector-add-py">
<div class="sphx-glr-download sphx-glr-download-python docutils container">
<p><a class="reference download internal" download="" href="../../_downloads/62d97d49a32414049819dd8bb8378080/01-vector-add.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">01-vector-add.py</span></code></a></p>

12
getting-started/tutorials/02-fused-softmax.html
View File

@@ -347,15 +347,15 @@ We will then compare its performance against (1) <code class="code docutils lite
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>softmax-performance:
N Triton Torch (native) Torch (jit)
0 256.0 512.000001 546.133347 273.066674
1 384.0 585.142862 585.142862 261.446801
1 384.0 585.142862 585.142862 267.130429
2 512.0 630.153853 606.814814 264.258068
3 640.0 682.666684 640.000002 265.974036
3 640.0 682.666684 640.000002 269.473696
4 768.0 702.171410 664.216187 273.066663
.. ... ... ... ...
93 12160.0 812.359066 405.755985 329.483481
94 12288.0 812.429770 415.222812 329.602681
93 12160.0 812.359066 406.179533 329.483481
94 12288.0 812.429770 415.661740 329.602681
95 12416.0 810.840807 412.149375 329.173158
96 12544.0 810.925276 412.971190 329.022957
96 12544.0 810.925276 412.546756 329.292871
97 12672.0 811.007961 412.097543 329.410251
[98 rows x 4 columns]
@@ -370,7 +370,7 @@ This means that – when temporary data is too large to fit entirely in the GPU
Note that our Triton kernel is not only faster than PyTorch’s CUDA kernel, it is also <strong>easier to read, understand and maintain</strong>.</p></li>
</ul>
</div></blockquote>
<p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 1 minutes 8.169 seconds)</p>
<p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 1 minutes 8.174 seconds)</p>
<div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-getting-started-tutorials-02-fused-softmax-py">
<div class="sphx-glr-download sphx-glr-download-python docutils container">
<p><a class="reference download internal" download="" href="../../_downloads/d91442ac2982c4e0cc3ab0f43534afbc/02-fused-softmax.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">02-fused-softmax.py</span></code></a></p>

54
getting-started/tutorials/03-matrix-multiplication.html
View File

@@ -476,42 +476,42 @@ tensor(True, device=&#39;cuda:0&#39;)
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>matmul-performance:
M cuBLAS ... Triton Triton (+ LeakyReLU)
0 128.0 0.455111 ... 0.512000 0.512000
1 256.0 2.730667 ... 2.978909 2.978909
2 384.0 7.372800 ... 8.507077 8.507077
3 512.0 14.563555 ... 15.420235 15.420235
4 640.0 22.260869 ... 23.272727 23.272727
1 256.0 2.978909 ... 2.978909 2.978909
2 384.0 7.372800 ... 7.899428 7.899428
3 512.0 14.563555 ... 16.384000 15.420235
4 640.0 22.260869 ... 24.380953 24.380953
5 768.0 32.768000 ... 34.028308 34.028308
6 896.0 39.025776 ... 39.025776 39.025776
7 1024.0 49.932191 ... 52.428801 52.428801
7 1024.0 51.150050 ... 52.428801 52.428801
8 1152.0 44.566925 ... 46.656000 46.656000
9 1280.0 51.200001 ... 56.109587 56.109587
10 1408.0 64.138541 ... 65.684049 65.684049
11 1536.0 80.430545 ... 76.106321 76.106321
12 1664.0 63.372618 ... 61.636381 61.636381
13 1792.0 72.983276 ... 69.379162 68.953520
14 1920.0 68.098521 ... 69.818184 69.818184
15 2048.0 73.584279 ... 75.234154 75.573044
16 2176.0 83.500614 ... 80.817862 80.494588
17 2304.0 68.251065 ... 73.051599 73.051599
18 2432.0 71.305746 ... 81.197876 81.197876
19 2560.0 77.833728 ... 76.740048 76.382283
20 2688.0 83.004501 ... 83.369354 85.051697
21 2816.0 81.067298 ... 75.982940 78.868366
22 2944.0 79.483304 ... 79.865439 79.230573
23 3072.0 81.589488 ... 83.146995 83.025078
24 3200.0 84.432717 ... 86.956520 89.385477
25 3328.0 84.003845 ... 82.843841 85.806075
26 3456.0 82.350937 ... 84.686523 84.508982
27 3584.0 84.111686 ... 92.032132 96.475743
28 3712.0 86.416391 ... 84.874549 88.170647
29 3840.0 85.005380 ... 87.771425 87.493673
30 3968.0 92.512459 ... 80.703662 84.797731
31 4096.0 93.727466 ... 90.995066 91.741443
11 1536.0 80.430545 ... 76.106321 75.296679
12 1664.0 63.372618 ... 62.061463 61.636381
13 1792.0 72.983276 ... 68.953520 68.533074
14 1920.0 69.120002 ... 68.435645 68.435645
15 2048.0 73.908442 ... 75.573044 75.234154
16 2176.0 83.500614 ... 80.173899 79.855747
17 2304.0 68.446623 ... 73.051599 72.607513
18 2432.0 71.125224 ... 81.197876 80.963875
19 2560.0 77.649287 ... 76.027843 76.740048
20 2688.0 83.552988 ... 83.186525 82.823267
21 2816.0 84.035084 ... 76.921000 79.733474
22 2944.0 82.102191 ... 80.122235 78.729910
23 3072.0 82.540970 ... 82.661468 82.661468
24 3200.0 84.432717 ... 89.385477 84.432717
25 3328.0 83.905938 ... 86.113988 86.528001
26 3456.0 82.015834 ... 83.545665 84.156124
27 3584.0 87.466332 ... 92.600816 84.988707
28 3712.0 85.163978 ... 82.902362 83.666116
29 3840.0 84.292684 ... 84.550462 85.070769
30 3968.0 89.921841 ... 87.472354 87.409694
31 4096.0 93.792965 ... 89.478485 90.260743
[32 rows x 5 columns]
</pre></div>
</div>
<p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 1 minutes 58.805 seconds)</p>
<p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 2 minutes 2.376 seconds)</p>
<div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-getting-started-tutorials-03-matrix-multiplication-py">
<div class="sphx-glr-download sphx-glr-download-python docutils container">
<p><a class="reference download internal" download="" href="../../_downloads/d5fee5b55a64e47f1b5724ec39adf171/03-matrix-multiplication.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">03-matrix-multiplication.py</span></code></a></p>

8
getting-started/tutorials/sg_execution_times.html
View File

@@ -174,7 +174,7 @@
<div class="section" id="computation-times">
<span id="sphx-glr-getting-started-tutorials-sg-execution-times"></span><h1>Computation times<a class="headerlink" href="#computation-times" title="Permalink to this headline"></a></h1>
<p><strong>03:18.042</strong> total execution time for <strong>getting-started_tutorials</strong> files:</p>
<p><strong>03:21.583</strong> total execution time for <strong>getting-started_tutorials</strong> files:</p>
<table class="docutils align-default">
<colgroup>
<col style="width: 85%" />
@@ -183,15 +183,15 @@
</colgroup>
<tbody>
<tr class="row-odd"><td><p><a class="reference internal" href="03-matrix-multiplication.html#sphx-glr-getting-started-tutorials-03-matrix-multiplication-py"><span class="std std-ref">Matrix Multiplication</span></a> (<code class="docutils literal notranslate"><span class="pre">03-matrix-multiplication.py</span></code>)</p></td>
<td><p>01:58.805</p></td>
<td><p>02:02.376</p></td>
<td><p>0.0 MB</p></td>
</tr>
<tr class="row-even"><td><p><a class="reference internal" href="02-fused-softmax.html#sphx-glr-getting-started-tutorials-02-fused-softmax-py"><span class="std std-ref">Fused Softmax</span></a> (<code class="docutils literal notranslate"><span class="pre">02-fused-softmax.py</span></code>)</p></td>
<td><p>01:08.169</p></td>
<td><p>01:08.174</p></td>
<td><p>0.0 MB</p></td>
</tr>
<tr class="row-odd"><td><p><a class="reference internal" href="01-vector-add.html#sphx-glr-getting-started-tutorials-01-vector-add-py"><span class="std std-ref">Vector Addition</span></a> (<code class="docutils literal notranslate"><span class="pre">01-vector-add.py</span></code>)</p></td>
<td><p>00:11.067</p></td>
<td><p>00:11.032</p></td>
<td><p>0.0 MB</p></td>
</tr>
</tbody>

BIN
objects.inv
View File

Binary file not shown.

32
programming-guide/chapter-2/related-work.html
View File

@@ -114,8 +114,8 @@
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#scheduling-languages">Scheduling Languages</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#id15">Advantages</a></li>
<li class="toctree-l3"><a class="reference internal" href="#id16">Limitations</a></li>
<li class="toctree-l3"><a class="reference internal" href="#id16">Advantages</a></li>
<li class="toctree-l3"><a class="reference internal" href="#id17">Limitations</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#references">References</a></li>
@@ -190,7 +190,7 @@
<div class="section" id="related-work">
<h1>Related Work<a class="headerlink" href="#related-work" title="Permalink to this headline"></a></h1>
<p>At first sight, Triton may seem like just yet another DSL for DNNs. The purpose of this section is to contextualize Triton and highlights its differences with the two leading approaches in this domain: polyhedral compilation and scheduling languages.</p>
<p>At first sight, Triton may seem like just yet another DSL for DNNs. The purpose of this section is to contextualize Triton and highlight its differences with the two leading approaches in this domain: polyhedral compilation and scheduling languages.</p>
<div class="section" id="polyhedral-compilation">
<h2>Polyhedral Compilation<a class="headerlink" href="#polyhedral-compilation" title="Permalink to this headline"></a></h2>
<p>Traditional compilers typically rely on intermediate representations, such as LLVM-IR <a class="reference internal" href="#lattner2004" id="id1"><span>[LATTNER2004]</span></a>, that encode control flow information using (un)conditional branches. This relatively low-level format makes it difficult to statically analyze the runtime behavior (e.g., cache misses) of input programs, and to automatically optimize loops accordingly through the use of tiling <a class="reference internal" href="#wolfe1989" id="id2"><span>[WOLFE1989]</span></a>, fusion <a class="reference internal" href="#darte1999" id="id3"><span>[DARTE1999]</span></a> and interchange <a class="reference internal" href="#allen1984" id="id4"><span>[ALLEN1984]</span></a>. To solve this issue, polyhedral compilers <a class="reference internal" href="#ancourt1991" id="id5"><span>[ANCOURT1991]</span></a> rely on program representations that have statically predictable control flow, thereby enabling aggressive compile-time program transformations for data locality and parallelism. Though this strategy has been adopted by many languages and compilers for DNNs such as Tiramisu <a class="reference internal" href="#baghdadi2021" id="id6"><span>[BAGHDADI2021]</span></a>, Tensor Comprehensions <a class="reference internal" href="#vasilache2018" id="id7"><span>[VASILACHE2018]</span></a>, Diesel <a class="reference internal" href="#elango2018" id="id8"><span>[ELANGO2018]</span></a> and the Affine dialect in MLIR <a class="reference internal" href="#lattner2019" id="id9"><span>[LATTNER2019]</span></a>, it also comes with a number of limitations that will be described later in this section.</p>
@@ -282,14 +282,14 @@ i &amp; j
<div class="section" id="limitations">
<h3>Limitations<a class="headerlink" href="#limitations" title="Permalink to this headline"></a></h3>
<p>Unfortunately, polyhedral compilers suffer from two major limitations that have prevented its adoption as a universal method for code generation in neural networks.</p>
<p>First, the set of possible program transformations $Omega = { Theta_S ~|~ S in text{program} }$ is large, and grows with the number of statements in the program as well as with the size of their iteration domain. Verifying the legality of each transformation can also require the resolution of complex integer linear programs, making polyhedral compilation very computationally expensive. To make matters worse, hardware properties (e.g., cache size, number of SMs) and contextual characteristics (e.g., input tensor shapes) also have to be taken into account by this framework, leading to expensive auto-tuning procedures <a class="reference internal" href="#sato2019" id="id12"><span>[SATO2019]</span></a>.</p>
<p>First, the set of possible program transformations <span class="math notranslate nohighlight">\(\Omega = \{ \Theta_S ~|~ S \in \text{program} \}\)</span> is large, and grows with the number of statements in the program as well as with the size of their iteration domain. Verifying the legality of each transformation can also require the resolution of complex integer linear programs, making polyhedral compilation very computationally expensive. To make matters worse, hardware properties (e.g., cache size, number of SMs) and contextual characteristics (e.g., input tensor shapes) also have to be taken into account by this framework, leading to expensive auto-tuning procedures <a class="reference internal" href="#sato2019" id="id12"><span>[SATO2019]</span></a>.</p>
<p>Second, the polyhedral framework is not very generally applicable; SCoPs are relatively common <a class="reference internal" href="#girbal2006" id="id13"><span>[GIRBAL2006]</span></a> but require loop bounds and array subscripts to be affine functions of loop indices, which typically only occurs in regular, dense computations. For this reason, this framework still has to be successfully applied to sparse – or even structured-sparse – neural networks, whose importance has been rapidly rising over the past few years.</p>
<p>On the other hand, blocked program representations advocated by this dissertation are less restricted in scope and can achieve close to peak performance using standard dataflow analysis.</p>
</div>
</div>
<div class="section" id="scheduling-languages">
<h2>Scheduling Languages<a class="headerlink" href="#scheduling-languages" title="Permalink to this headline"></a></h2>
<p>Separation of concerns cite{dijkstra82} is a well-known design principle in computer science: programs should be decomposed into modular layers of abstraction that separate the semantics of their algorithms from the details of their implementation. Systems like Halide and TVM push this philosophy one step further, and enforce this separation at the grammatical level through the use of a <strong>scheduling language</strong>. The benefits of this methodology are particularly visible in the case of matrix multiplication, where, as one can see below, the definition of the algorithm (Line 1-7) is completely disjoint from its implementation (Line 8-16), meaning that both can be maintained, optimized and distributed independently.</p>
<p>Separation of concerns <a class="reference internal" href="#dijkstra82" id="id14"><span>[DIJKSTRA82]</span></a> is a well-known design principle in computer science: programs should be decomposed into modular layers of abstraction that separate the semantics of their algorithms from the details of their implementation. Systems like Halide and TVM push this philosophy one step further, and enforce this separation at the grammatical level through the use of a <strong>scheduling language</strong>. The benefits of this methodology are particularly visible in the case of matrix multiplication, where, as one can see below, the definition of the algorithm (Line 1-7) is completely disjoint from its implementation (Line 8-16), meaning that both can be maintained, optimized and distributed independently.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="linenos"> 1</span><span class="o">//</span> <span class="n">algorithm</span>
<span class="linenos"> 2</span><span class="n">Var</span> <span class="n">x</span><span class="p">(</span><span class="s2">&quot;x&quot;</span><span class="p">),</span> <span class="n">y</span><span class="p">(</span><span class="s2">&quot;y&quot;</span><span class="p">);</span>
<span class="linenos"> 3</span><span class="n">Func</span> <span class="n">matmul</span><span class="p">(</span><span class="s2">&quot;matmul&quot;</span><span class="p">);</span>
@@ -308,15 +308,15 @@ i &amp; j
<span class="linenos">16</span> <span class="o">.</span><span class="n">parallel</span><span class="p">(</span><span class="n">y</span><span class="p">)</span><span class="o">.</span><span class="n">vectorize</span><span class="p">(</span><span class="n">xii</span><span class="p">)</span><span class="o">.</span><span class="n">unroll</span><span class="p">(</span><span class="n">xi</span><span class="p">)</span><span class="o">.</span><span class="n">unroll</span><span class="p">(</span><span class="n">yii</span><span class="p">);</span>
</pre></div>
</div>
<p>The resulting code may however not be completely portable, as schedules can sometimes rely on execution models (e.g., SPMD) or hardware intrinsics (e.g., matrix-multiply-accumulate) that are not widely available. This issue can be mitigated by auto-scheduling mechanisms <a class="reference internal" href="#mullapudi2016" id="id14"><span>[MULLAPUDI2016]</span></a>.</p>
<div class="section" id="id15">
<h3>Advantages<a class="headerlink" href="#id15" title="Permalink to this headline"></a></h3>
<p>The resulting code may however not be completely portable, as schedules can sometimes rely on execution models (e.g., SPMD) or hardware intrinsics (e.g., matrix-multiply-accumulate) that are not widely available. This issue can be mitigated by auto-scheduling mechanisms <a class="reference internal" href="#mullapudi2016" id="id15"><span>[MULLAPUDI2016]</span></a>.</p>
<div class="section" id="id16">
<h3>Advantages<a class="headerlink" href="#id16" title="Permalink to this headline"></a></h3>
<p>The main advantage of this approach is that it allows programmers to write an algorithm <em>only once</em>, and focus on performance optimization separately. It makes it possible to manually specify optimizations that a polyhedral compiler wouldn’t be able to figure out automatically using static data-flow analysis.</p>
<p>Scheduling languages are, without a doubt, one of the most popular approaches for neural network code generation. The most popular system for this purpose is probably TVM, which provides good performance across a wide range of platforms as well as built-in automatic scheduling mechanisms.</p>
</div>
<div class="section" id="id16">
<h3>Limitations<a class="headerlink" href="#id16" title="Permalink to this headline"></a></h3>
<p>This ease-of-development comes at a cost. First of all, existing systems that follow this paradigm tend to be noticeably slower than Triton on modern hardware when applicable (e.g., V100/A100 tensor cores w/ equal tile sizes). I do believe that this is not a fundamental issue of scheduling languages – in the sense that it could probably be solved with more efforts – but it could mean that these systems are harder to engineer. More importantly, existing scheduling languages generate loops whose bounds and increments cannot depend on surrounding loop indice without at least imposing severe constraints on possible schedules – if not breaking the system entirely. This is problematic for sparse com-putations, whose iteration spaces may be irregular.</p>
<div class="section" id="id17">
<h3>Limitations<a class="headerlink" href="#id17" title="Permalink to this headline"></a></h3>
<p>This ease-of-development comes at a cost. First of all, existing systems that follow this paradigm tend to be noticeably slower than Triton on modern hardware when applicable (e.g., V100/A100 tensor cores w/ equal tile sizes). I do believe that this is not a fundamental issue of scheduling languages – in the sense that it could probably be solved with more efforts – but it could mean that these systems are harder to engineer. More importantly, existing scheduling languages generate loops whose bounds and increments cannot depend on surrounding loop indice without at least imposing severe constraints on possible schedules – if not breaking the system entirely. This is problematic for sparse computations, whose iteration spaces may be irregular.</p>
<table class="colwidths-given docutils align-default">
<colgroup>
<col style="width: 50%" />
@@ -402,7 +402,15 @@ i &amp; j
<li><p>Girbal et al., “Semi-Automatic Composition of Loop Transformations for Deep Parallelism and Memory Hierarchies”, International Journal of Parallel Programming 2006</p></li>
</ol>
</dd>
<dt class="label" id="mullapudi2016"><span class="brackets"><a class="fn-backref" href="#id14">MULLAPUDI2016</a></span></dt>
<dt class="label" id="dijkstra82"><span class="brackets"><a class="fn-backref" href="#id14">DIJKSTRA82</a></span></dt>
<dd><ol class="upperalpha simple" start="5">
<li><ol class="upperalpha simple" start="23">
<li><p>Dijkstra et al., “On the role of scientific thought”, Selected writings on computing: a personal perspective 1982</p></li>
</ol>
</li>
</ol>
</dd>
<dt class="label" id="mullapudi2016"><span class="brackets"><a class="fn-backref" href="#id15">MULLAPUDI2016</a></span></dt>
<dd><ol class="upperalpha simple" start="18">
<li><p>Mullapudi et al., “Automatically scheduling halide image processing pipelines”, TOG 2016</p></li>
</ol>

2
searchindex.js
View File

File diff suppressed because one or more lines are too long