[GH-PAGES] Updated website

This commit is contained in:
Philippe Tillet
2022-02-22 00:40:26 +00:00
parent 5a89721fe3
commit c9eba0bba7
158 changed files with 260 additions and 260 deletions

View File

@@ -1,4 +1,4 @@
# Sphinx build info version 1 # Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: 803119500a0a5b801d98e2d05bc29225 config: 01fd3208581800c33b1bfbbe059acfe5
tags: 645f666f9bcd5a90fca523b33c5a78b7 tags: 645f666f9bcd5a90fca523b33c5a78b7

Binary file not shown.

Binary file not shown.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 16 KiB

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 37 KiB

After

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 24 KiB

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 34 KiB

After

Width:  |  Height:  |  Size: 33 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 32 KiB

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 20 KiB

View File

@@ -235,10 +235,10 @@ We can now run the decorated function above. Pass `print_data=True` to see the p
0 4096.0 9.600000 9.600000 0 4096.0 9.600000 9.600000
1 8192.0 19.200000 19.200000 1 8192.0 19.200000 19.200000
2 16384.0 38.400001 38.400001 2 16384.0 38.400001 38.400001
3 32768.0 76.800002 76.800002 3 32768.0 63.999998 76.800002
4 65536.0 127.999995 127.999995 4 65536.0 127.999995 127.999995
5 131072.0 219.428568 219.428568 5 131072.0 219.428568 219.428568
6 262144.0 341.333321 384.000001 6 262144.0 384.000001 384.000001
7 524288.0 472.615390 472.615390 7 524288.0 472.615390 472.615390
8 1048576.0 614.400016 614.400016 8 1048576.0 614.400016 614.400016
9 2097152.0 722.823517 722.823517 9 2097152.0 722.823517 722.823517
@@ -255,7 +255,7 @@ We can now run the decorated function above. Pass `print_data=True` to see the p
.. rst-class:: sphx-glr-timing .. rst-class:: sphx-glr-timing
**Total running time of the script:** ( 1 minutes 38.111 seconds) **Total running time of the script:** ( 1 minutes 44.030 seconds)
.. _sphx_glr_download_getting-started_tutorials_01-vector-add.py: .. _sphx_glr_download_getting-started_tutorials_01-vector-add.py:

View File

@@ -284,10 +284,10 @@ We will then compare its performance against (1) :code:`torch.softmax` and (2) t
3 640.0 706.206879 640.000002 160.000000 3 640.0 706.206879 640.000002 160.000000
4 768.0 722.823517 664.216187 162.754967 4 768.0 722.823517 664.216187 162.754967
.. ... ... ... ... .. ... ... ... ...
93 12160.0 814.058574 406.179533 199.038365 93 12160.0 814.058574 406.179533 198.936606
94 12288.0 814.111783 415.222812 199.298541 94 12288.0 814.111783 415.661740 199.298541
95 12416.0 814.163950 412.149375 198.854847 95 12416.0 814.163950 412.149375 198.854847
96 12544.0 814.214963 412.971190 199.111113 96 12544.0 814.214963 412.971190 199.012395
97 12672.0 814.265046 412.097543 199.069228 97 12672.0 814.265046 412.097543 199.069228
[98 rows x 4 columns] [98 rows x 4 columns]
@@ -306,7 +306,7 @@ In the above plot, we can see that:
.. rst-class:: sphx-glr-timing .. rst-class:: sphx-glr-timing
**Total running time of the script:** ( 3 minutes 19.799 seconds) **Total running time of the script:** ( 3 minutes 20.262 seconds)
.. _sphx_glr_download_getting-started_tutorials_02-fused-softmax.py: .. _sphx_glr_download_getting-started_tutorials_02-fused-softmax.py:

View File

@@ -458,37 +458,37 @@ We can now compare the performance of our kernel against that of cuBLAS. Here we
matmul-performance: matmul-performance:
M cuBLAS ... Triton Triton (+ LeakyReLU) M cuBLAS ... Triton Triton (+ LeakyReLU)
0 256.0 2.730667 ... 3.276800 2.978909 0 256.0 2.978909 ... 2.978909 2.978909
1 384.0 7.372800 ... 8.507077 8.507077 1 384.0 7.372800 ... 8.507077 7.899428
2 512.0 14.563555 ... 16.384000 15.420235 2 512.0 14.563555 ... 16.384000 15.420235
3 640.0 22.260869 ... 24.380953 24.380953 3 640.0 22.260869 ... 24.380953 24.380953
4 768.0 32.768000 ... 34.028308 34.028308 4 768.0 32.768000 ... 34.028308 34.028308
5 896.0 37.971025 ... 40.140799 39.025776 5 896.0 39.025776 ... 39.025776 39.025776
6 1024.0 49.932191 ... 52.428801 52.428801 6 1024.0 49.932191 ... 52.428801 52.428801
7 1152.0 45.242181 ... 46.656000 46.656000 7 1152.0 45.242181 ... 46.656000 46.656000
8 1280.0 51.200001 ... 56.888887 56.888887 8 1280.0 51.200001 ... 56.888887 56.888887
9 1408.0 64.138541 ... 67.305878 66.485074 9 1408.0 64.138541 ... 67.305878 66.485074
10 1536.0 80.430545 ... 79.526831 78.643199 10 1536.0 79.526831 ... 79.526831 79.526831
11 1664.0 62.929456 ... 62.492442 62.492442 11 1664.0 62.929456 ... 62.929456 62.492442
12 1792.0 72.512412 ... 72.047592 72.047592 12 1792.0 72.983276 ... 72.047592 71.588687
13 1920.0 68.776119 ... 70.172588 70.172588 13 1920.0 69.120002 ... 70.172588 69.818184
14 2048.0 73.262953 ... 76.608294 76.260072 14 2048.0 73.262953 ... 76.608294 76.260072
15 2176.0 83.155572 ... 85.815126 85.632545 15 2176.0 83.155572 ... 85.998493 85.632545
16 2304.0 68.251065 ... 76.809875 76.809875 16 2304.0 68.251065 ... 77.057651 76.809875
17 2432.0 71.305746 ... 84.877538 85.134737 17 2432.0 71.487187 ... 84.877538 84.367759
18 2560.0 77.283019 ... 80.908642 81.108913 18 2560.0 78.019048 ... 81.108913 80.511054
19 2688.0 83.369354 ... 89.888756 88.628636 19 2688.0 83.186525 ... 90.102270 89.044736
20 2816.0 80.469019 ... 82.916747 81.369790 20 2816.0 81.674548 ... 82.759409 82.759409
21 2944.0 81.967162 ... 81.698415 82.237674 21 2944.0 82.921853 ... 81.832567 81.967162
22 3072.0 82.782312 ... 88.612060 88.335577 22 3072.0 81.707223 ... 88.060814 88.750943
23 3200.0 84.656085 ... 95.380032 94.395283 23 3200.0 80.503145 ... 87.791493 91.888011
24 3328.0 83.419811 ... 81.438120 85.602017 24 3328.0 79.901550 ... 84.895397 84.695641
25 3456.0 81.849303 ... 90.382926 91.097818 25 3456.0 81.849303 ... 91.407671 91.200871
26 3584.0 84.033077 ... 90.458141 96.579370 26 3584.0 84.427601 ... 90.458141 96.579370
27 3712.0 85.970176 ... 85.530838 88.170647 27 3712.0 85.748791 ... 86.980708 87.170458
28 3840.0 81.168440 ... 85.531326 91.777595 28 3840.0 79.334289 ... 91.777595 84.292684
29 3968.0 88.040360 ... 90.321193 84.094627 29 3968.0 90.859224 ... 84.915752 90.994735
30 4096.0 92.500158 ... 87.381330 88.069377 30 4096.0 86.424811 ... 84.735086 91.678778
[31 rows x 5 columns] [31 rows x 5 columns]
@@ -498,7 +498,7 @@ We can now compare the performance of our kernel against that of cuBLAS. Here we
.. rst-class:: sphx-glr-timing .. rst-class:: sphx-glr-timing
**Total running time of the script:** ( 6 minutes 1.523 seconds) **Total running time of the script:** ( 5 minutes 59.250 seconds)
.. _sphx_glr_download_getting-started_tutorials_03-matrix-multiplication.py: .. _sphx_glr_download_getting-started_tutorials_03-matrix-multiplication.py:

View File

@@ -240,7 +240,7 @@ References
.. rst-class:: sphx-glr-timing .. rst-class:: sphx-glr-timing
**Total running time of the script:** ( 0 minutes 0.477 seconds) **Total running time of the script:** ( 0 minutes 0.482 seconds)
.. _sphx_glr_download_getting-started_tutorials_04-low-memory-dropout.py: .. _sphx_glr_download_getting-started_tutorials_04-low-memory-dropout.py:

View File

@@ -38,36 +38,36 @@ Layer Normalization
layer-norm-backward: layer-norm-backward:
N Triton Torch Apex N Triton Torch Apex
0 1024.0 307.200008 98.303995 303.407414 0 1024.0 311.088617 98.303995 307.200008
1 1536.0 347.773587 134.050910 341.333333 1 1536.0 347.773587 134.540150 344.523365
2 2048.0 420.102553 161.684218 325.509933 2 2048.0 423.724127 161.684218 325.509933
3 2560.0 458.507457 181.775141 326.808501 3 2560.0 458.507457 181.775141 326.808501
4 3072.0 508.468972 192.501302 317.793096 4 3072.0 511.999982 192.501302 317.793096
5 3584.0 551.384634 208.271186 310.527060 5 3584.0 551.384634 208.271186 311.652167
6 4096.0 564.965515 220.412561 294.323343 6 4096.0 568.231237 220.412561 295.207204
7 4608.0 504.986315 232.825259 291.031570 7 4608.0 504.986315 232.825259 292.571431
8 5120.0 529.655159 242.845844 287.775181 8 5120.0 529.655159 242.845844 288.450695
9 5632.0 542.843364 243.545956 288.820505 9 5632.0 545.032265 243.545956 289.438969
10 6144.0 546.133354 248.242431 286.322318 10 6144.0 546.133354 248.661056 286.879370
11 6656.0 534.260858 256.000009 285.767438 11 6656.0 534.260858 256.000009 285.767438
12 7168.0 507.469040 260.260201 286.242939 12 7168.0 507.469040 260.654538 286.242939
13 7680.0 479.999983 262.564106 279.272719 13 7680.0 479.999983 262.564106 278.850215
14 8192.0 462.607053 267.130429 284.526763 14 8192.0 463.698115 267.130429 285.352682
15 8704.0 417.791980 267.472468 284.987724 15 8704.0 417.791980 267.815384 284.987724
16 9216.0 431.157889 272.394084 288.751954 16 9216.0 431.157889 272.729961 288.751954
17 9728.0 438.857162 280.110377 289.667485 17 9728.0 438.857162 280.278512 290.027323
18 10240.0 449.287041 286.433562 289.811322 18 10240.0 450.109870 286.433562 290.153487
19 10752.0 426.525614 247.172406 290.594591 19 10752.0 427.231788 246.935876 290.922209
20 11264.0 426.397479 245.536784 286.676558 20 11264.0 426.397479 245.760001 286.676558
21 11776.0 423.089806 249.667843 288.981596 21 11776.0 422.457417 249.888595 288.981596
22 12288.0 419.504980 254.673582 294.323369 22 12288.0 419.504980 254.673582 294.323369
23 12800.0 414.016170 253.465340 288.180121 23 12800.0 414.016170 253.674644 288.450715
24 13312.0 411.181478 252.759501 289.916513 24 13312.0 411.181478 252.959629 290.179836
25 13824.0 404.112047 257.190689 292.056329 25 13824.0 404.604870 257.190689 292.056329
26 14336.0 393.665895 254.297107 286.719986 26 14336.0 393.215988 254.485198 286.719986
27 14848.0 385.245405 257.665934 289.012175 27 14848.0 385.245405 257.665934 289.246765
28 15360.0 373.874218 257.790220 286.211174 28 15360.0 373.874218 257.790220 286.211174
29 15872.0 370.913333 261.626369 289.899545 29 15872.0 371.274849 261.626369 290.120338
@@ -339,7 +339,7 @@ Layer Normalization
.. rst-class:: sphx-glr-timing .. rst-class:: sphx-glr-timing
**Total running time of the script:** ( 2 minutes 12.404 seconds) **Total running time of the script:** ( 2 minutes 12.441 seconds)
.. _sphx_glr_download_getting-started_tutorials_05-layer-norm.py: .. _sphx_glr_download_getting-started_tutorials_05-layer-norm.py:

View File

@@ -5,16 +5,16 @@
Computation times Computation times
================= =================
**13:12.314** total execution time for **getting-started_tutorials** files: **13:16.466** total execution time for **getting-started_tutorials** files:
+---------------------------------------------------------------------------------------------------------+-----------+--------+ +---------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_getting-started_tutorials_03-matrix-multiplication.py` (``03-matrix-multiplication.py``) | 06:01.523 | 0.0 MB | | :ref:`sphx_glr_getting-started_tutorials_03-matrix-multiplication.py` (``03-matrix-multiplication.py``) | 05:59.250 | 0.0 MB |
+---------------------------------------------------------------------------------------------------------+-----------+--------+ +---------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_getting-started_tutorials_02-fused-softmax.py` (``02-fused-softmax.py``) | 03:19.799 | 0.0 MB | | :ref:`sphx_glr_getting-started_tutorials_02-fused-softmax.py` (``02-fused-softmax.py``) | 03:20.262 | 0.0 MB |
+---------------------------------------------------------------------------------------------------------+-----------+--------+ +---------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_getting-started_tutorials_05-layer-norm.py` (``05-layer-norm.py``) | 02:12.404 | 0.0 MB | | :ref:`sphx_glr_getting-started_tutorials_05-layer-norm.py` (``05-layer-norm.py``) | 02:12.441 | 0.0 MB |
+---------------------------------------------------------------------------------------------------------+-----------+--------+ +---------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_getting-started_tutorials_01-vector-add.py` (``01-vector-add.py``) | 01:38.111 | 0.0 MB | | :ref:`sphx_glr_getting-started_tutorials_01-vector-add.py` (``01-vector-add.py``) | 01:44.030 | 0.0 MB |
+---------------------------------------------------------------------------------------------------------+-----------+--------+ +---------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_getting-started_tutorials_04-low-memory-dropout.py` (``04-low-memory-dropout.py``) | 00:00.477 | 0.0 MB | | :ref:`sphx_glr_getting-started_tutorials_04-low-memory-dropout.py` (``04-low-memory-dropout.py``) | 00:00.482 | 0.0 MB |
+---------------------------------------------------------------------------------------------------------+-----------+--------+ +---------------------------------------------------------------------------------------------------------+-----------+--------+

View File

@@ -325,10 +325,10 @@ for different problem sizes.</p>
0 4096.0 9.600000 9.600000 0 4096.0 9.600000 9.600000
1 8192.0 19.200000 19.200000 1 8192.0 19.200000 19.200000
2 16384.0 38.400001 38.400001 2 16384.0 38.400001 38.400001
3 32768.0 76.800002 76.800002 3 32768.0 63.999998 76.800002
4 65536.0 127.999995 127.999995 4 65536.0 127.999995 127.999995
5 131072.0 219.428568 219.428568 5 131072.0 219.428568 219.428568
6 262144.0 341.333321 384.000001 6 262144.0 384.000001 384.000001
7 524288.0 472.615390 472.615390 7 524288.0 472.615390 472.615390
8 1048576.0 614.400016 614.400016 8 1048576.0 614.400016 614.400016
9 2097152.0 722.823517 722.823517 9 2097152.0 722.823517 722.823517
@@ -340,7 +340,7 @@ for different problem sizes.</p>
15 134217728.0 849.737435 850.656574 15 134217728.0 849.737435 850.656574
</pre></div> </pre></div>
</div> </div>
<p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 1 minutes 38.111 seconds)</p> <p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 1 minutes 44.030 seconds)</p>
<div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-getting-started-tutorials-01-vector-add-py"> <div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-getting-started-tutorials-01-vector-add-py">
<div class="sphx-glr-download sphx-glr-download-python docutils container"> <div class="sphx-glr-download sphx-glr-download-python docutils container">
<p><a class="reference download internal" download="" href="../../_downloads/62d97d49a32414049819dd8bb8378080/01-vector-add.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">01-vector-add.py</span></code></a></p> <p><a class="reference download internal" download="" href="../../_downloads/62d97d49a32414049819dd8bb8378080/01-vector-add.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">01-vector-add.py</span></code></a></p>

View File

@@ -375,10 +375,10 @@ We will then compare its performance against (1) <code class="code docutils lite
3 640.0 706.206879 640.000002 160.000000 3 640.0 706.206879 640.000002 160.000000
4 768.0 722.823517 664.216187 162.754967 4 768.0 722.823517 664.216187 162.754967
.. ... ... ... ... .. ... ... ... ...
93 12160.0 814.058574 406.179533 199.038365 93 12160.0 814.058574 406.179533 198.936606
94 12288.0 814.111783 415.222812 199.298541 94 12288.0 814.111783 415.661740 199.298541
95 12416.0 814.163950 412.149375 198.854847 95 12416.0 814.163950 412.149375 198.854847
96 12544.0 814.214963 412.971190 199.111113 96 12544.0 814.214963 412.971190 199.012395
97 12672.0 814.265046 412.097543 199.069228 97 12672.0 814.265046 412.097543 199.069228
[98 rows x 4 columns] [98 rows x 4 columns]
@@ -392,7 +392,7 @@ We will then compare its performance against (1) <code class="code docutils lite
Note however that the PyTorch <cite>softmax</cite> operation is more general and will works on tensors of any shape.</p></li> Note however that the PyTorch <cite>softmax</cite> operation is more general and will works on tensors of any shape.</p></li>
</ul> </ul>
</div></blockquote> </div></blockquote>
<p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 3 minutes 19.799 seconds)</p> <p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 3 minutes 20.262 seconds)</p>
<div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-getting-started-tutorials-02-fused-softmax-py"> <div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-getting-started-tutorials-02-fused-softmax-py">
<div class="sphx-glr-download sphx-glr-download-python docutils container"> <div class="sphx-glr-download sphx-glr-download-python docutils container">
<p><a class="reference download internal" download="" href="../../_downloads/d91442ac2982c4e0cc3ab0f43534afbc/02-fused-softmax.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">02-fused-softmax.py</span></code></a></p> <p><a class="reference download internal" download="" href="../../_downloads/d91442ac2982c4e0cc3ab0f43534afbc/02-fused-softmax.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">02-fused-softmax.py</span></code></a></p>

View File

@@ -564,42 +564,42 @@ torch_output=tensor([[ 1.1045, -36.9688, 31.4688, ..., -11.3906, 24.4531, -3
<p class="sphx-glr-script-out">Out:</p> <p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>matmul-performance: <div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>matmul-performance:
M cuBLAS ... Triton Triton (+ LeakyReLU) M cuBLAS ... Triton Triton (+ LeakyReLU)
0 256.0 2.730667 ... 3.276800 2.978909 0 256.0 2.978909 ... 2.978909 2.978909
1 384.0 7.372800 ... 8.507077 8.507077 1 384.0 7.372800 ... 8.507077 7.899428
2 512.0 14.563555 ... 16.384000 15.420235 2 512.0 14.563555 ... 16.384000 15.420235
3 640.0 22.260869 ... 24.380953 24.380953 3 640.0 22.260869 ... 24.380953 24.380953
4 768.0 32.768000 ... 34.028308 34.028308 4 768.0 32.768000 ... 34.028308 34.028308
5 896.0 37.971025 ... 40.140799 39.025776 5 896.0 39.025776 ... 39.025776 39.025776
6 1024.0 49.932191 ... 52.428801 52.428801 6 1024.0 49.932191 ... 52.428801 52.428801
7 1152.0 45.242181 ... 46.656000 46.656000 7 1152.0 45.242181 ... 46.656000 46.656000
8 1280.0 51.200001 ... 56.888887 56.888887 8 1280.0 51.200001 ... 56.888887 56.888887
9 1408.0 64.138541 ... 67.305878 66.485074 9 1408.0 64.138541 ... 67.305878 66.485074
10 1536.0 80.430545 ... 79.526831 78.643199 10 1536.0 79.526831 ... 79.526831 79.526831
11 1664.0 62.929456 ... 62.492442 62.492442 11 1664.0 62.929456 ... 62.929456 62.492442
12 1792.0 72.512412 ... 72.047592 72.047592 12 1792.0 72.983276 ... 72.047592 71.588687
13 1920.0 68.776119 ... 70.172588 70.172588 13 1920.0 69.120002 ... 70.172588 69.818184
14 2048.0 73.262953 ... 76.608294 76.260072 14 2048.0 73.262953 ... 76.608294 76.260072
15 2176.0 83.155572 ... 85.815126 85.632545 15 2176.0 83.155572 ... 85.998493 85.632545
16 2304.0 68.251065 ... 76.809875 76.809875 16 2304.0 68.251065 ... 77.057651 76.809875
17 2432.0 71.305746 ... 84.877538 85.134737 17 2432.0 71.487187 ... 84.877538 84.367759
18 2560.0 77.283019 ... 80.908642 81.108913 18 2560.0 78.019048 ... 81.108913 80.511054
19 2688.0 83.369354 ... 89.888756 88.628636 19 2688.0 83.186525 ... 90.102270 89.044736
20 2816.0 80.469019 ... 82.916747 81.369790 20 2816.0 81.674548 ... 82.759409 82.759409
21 2944.0 81.967162 ... 81.698415 82.237674 21 2944.0 82.921853 ... 81.832567 81.967162
22 3072.0 82.782312 ... 88.612060 88.335577 22 3072.0 81.707223 ... 88.060814 88.750943
23 3200.0 84.656085 ... 95.380032 94.395283 23 3200.0 80.503145 ... 87.791493 91.888011
24 3328.0 83.419811 ... 81.438120 85.602017 24 3328.0 79.901550 ... 84.895397 84.695641
25 3456.0 81.849303 ... 90.382926 91.097818 25 3456.0 81.849303 ... 91.407671 91.200871
26 3584.0 84.033077 ... 90.458141 96.579370 26 3584.0 84.427601 ... 90.458141 96.579370
27 3712.0 85.970176 ... 85.530838 88.170647 27 3712.0 85.748791 ... 86.980708 87.170458
28 3840.0 81.168440 ... 85.531326 91.777595 28 3840.0 79.334289 ... 91.777595 84.292684
29 3968.0 88.040360 ... 90.321193 84.094627 29 3968.0 90.859224 ... 84.915752 90.994735
30 4096.0 92.500158 ... 87.381330 88.069377 30 4096.0 86.424811 ... 84.735086 91.678778
[31 rows x 5 columns] [31 rows x 5 columns]
</pre></div> </pre></div>
</div> </div>
<p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 6 minutes 1.523 seconds)</p> <p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 5 minutes 59.250 seconds)</p>
<div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-getting-started-tutorials-03-matrix-multiplication-py"> <div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-getting-started-tutorials-03-matrix-multiplication-py">
<div class="sphx-glr-download sphx-glr-download-python docutils container"> <div class="sphx-glr-download sphx-glr-download-python docutils container">
<p><a class="reference download internal" download="" href="../../_downloads/d5fee5b55a64e47f1b5724ec39adf171/03-matrix-multiplication.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">03-matrix-multiplication.py</span></code></a></p> <p><a class="reference download internal" download="" href="../../_downloads/d5fee5b55a64e47f1b5724ec39adf171/03-matrix-multiplication.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">03-matrix-multiplication.py</span></code></a></p>

View File

@@ -372,7 +372,7 @@ to explore the <cite>triton/language/random</cite> folder!</p>
<dd><p>Nitish Srivastava and Geoffrey Hinton and Alex Krizhevsky and Ilya Sutskever and Ruslan Salakhutdinov, “Dropout: A Simple Way to Prevent Neural Networks from Overfitting”, JMLR 2014</p> <dd><p>Nitish Srivastava and Geoffrey Hinton and Alex Krizhevsky and Ilya Sutskever and Ruslan Salakhutdinov, “Dropout: A Simple Way to Prevent Neural Networks from Overfitting”, JMLR 2014</p>
</dd> </dd>
</dl> </dl>
<p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 0 minutes 0.477 seconds)</p> <p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 0 minutes 0.482 seconds)</p>
<div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-getting-started-tutorials-04-low-memory-dropout-py"> <div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-getting-started-tutorials-04-low-memory-dropout-py">
<div class="sphx-glr-download sphx-glr-download-python docutils container"> <div class="sphx-glr-download sphx-glr-download-python docutils container">
<p><a class="reference download internal" download="" href="../../_downloads/c9aed78977a4c05741d675a38dde3d7d/04-low-memory-dropout.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">04-low-memory-dropout.py</span></code></a></p> <p><a class="reference download internal" download="" href="../../_downloads/c9aed78977a4c05741d675a38dde3d7d/04-low-memory-dropout.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">04-low-memory-dropout.py</span></code></a></p>

View File

@@ -194,36 +194,36 @@ to download the full example code</p>
<p class="sphx-glr-script-out">Out:</p> <p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>layer-norm-backward: <div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>layer-norm-backward:
N Triton Torch Apex N Triton Torch Apex
0 1024.0 307.200008 98.303995 303.407414 0 1024.0 311.088617 98.303995 307.200008
1 1536.0 347.773587 134.050910 341.333333 1 1536.0 347.773587 134.540150 344.523365
2 2048.0 420.102553 161.684218 325.509933 2 2048.0 423.724127 161.684218 325.509933
3 2560.0 458.507457 181.775141 326.808501 3 2560.0 458.507457 181.775141 326.808501
4 3072.0 508.468972 192.501302 317.793096 4 3072.0 511.999982 192.501302 317.793096
5 3584.0 551.384634 208.271186 310.527060 5 3584.0 551.384634 208.271186 311.652167
6 4096.0 564.965515 220.412561 294.323343 6 4096.0 568.231237 220.412561 295.207204
7 4608.0 504.986315 232.825259 291.031570 7 4608.0 504.986315 232.825259 292.571431
8 5120.0 529.655159 242.845844 287.775181 8 5120.0 529.655159 242.845844 288.450695
9 5632.0 542.843364 243.545956 288.820505 9 5632.0 545.032265 243.545956 289.438969
10 6144.0 546.133354 248.242431 286.322318 10 6144.0 546.133354 248.661056 286.879370
11 6656.0 534.260858 256.000009 285.767438 11 6656.0 534.260858 256.000009 285.767438
12 7168.0 507.469040 260.260201 286.242939 12 7168.0 507.469040 260.654538 286.242939
13 7680.0 479.999983 262.564106 279.272719 13 7680.0 479.999983 262.564106 278.850215
14 8192.0 462.607053 267.130429 284.526763 14 8192.0 463.698115 267.130429 285.352682
15 8704.0 417.791980 267.472468 284.987724 15 8704.0 417.791980 267.815384 284.987724
16 9216.0 431.157889 272.394084 288.751954 16 9216.0 431.157889 272.729961 288.751954
17 9728.0 438.857162 280.110377 289.667485 17 9728.0 438.857162 280.278512 290.027323
18 10240.0 449.287041 286.433562 289.811322 18 10240.0 450.109870 286.433562 290.153487
19 10752.0 426.525614 247.172406 290.594591 19 10752.0 427.231788 246.935876 290.922209
20 11264.0 426.397479 245.536784 286.676558 20 11264.0 426.397479 245.760001 286.676558
21 11776.0 423.089806 249.667843 288.981596 21 11776.0 422.457417 249.888595 288.981596
22 12288.0 419.504980 254.673582 294.323369 22 12288.0 419.504980 254.673582 294.323369
23 12800.0 414.016170 253.465340 288.180121 23 12800.0 414.016170 253.674644 288.450715
24 13312.0 411.181478 252.759501 289.916513 24 13312.0 411.181478 252.959629 290.179836
25 13824.0 404.112047 257.190689 292.056329 25 13824.0 404.604870 257.190689 292.056329
26 14336.0 393.665895 254.297107 286.719986 26 14336.0 393.215988 254.485198 286.719986
27 14848.0 385.245405 257.665934 289.012175 27 14848.0 385.245405 257.665934 289.246765
28 15360.0 373.874218 257.790220 286.211174 28 15360.0 373.874218 257.790220 286.211174
29 15872.0 370.913333 261.626369 289.899545 29 15872.0 371.274849 261.626369 290.120338
</pre></div> </pre></div>
</div> </div>
<div class="line-block"> <div class="line-block">
@@ -487,7 +487,7 @@ to download the full example code</p>
<span class="n">bench_layer_norm</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">save_path</span><span class="o">=</span><span class="s1">&#39;.&#39;</span><span class="p">,</span> <span class="n">print_data</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="n">bench_layer_norm</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">save_path</span><span class="o">=</span><span class="s1">&#39;.&#39;</span><span class="p">,</span> <span class="n">print_data</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</pre></div> </pre></div>
</div> </div>
<p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 2 minutes 12.404 seconds)</p> <p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 2 minutes 12.441 seconds)</p>
<div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-getting-started-tutorials-05-layer-norm-py"> <div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-getting-started-tutorials-05-layer-norm-py">
<div class="sphx-glr-download sphx-glr-download-python docutils container"> <div class="sphx-glr-download sphx-glr-download-python docutils container">
<p><a class="reference download internal" download="" href="../../_downloads/935c0dd0fbeb4b2e69588471cbb2d4b2/05-layer-norm.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">05-layer-norm.py</span></code></a></p> <p><a class="reference download internal" download="" href="../../_downloads/935c0dd0fbeb4b2e69588471cbb2d4b2/05-layer-norm.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">05-layer-norm.py</span></code></a></p>

View File

@@ -174,7 +174,7 @@
<div class="section" id="computation-times"> <div class="section" id="computation-times">
<span id="sphx-glr-getting-started-tutorials-sg-execution-times"></span><h1>Computation times<a class="headerlink" href="#computation-times" title="Permalink to this headline"></a></h1> <span id="sphx-glr-getting-started-tutorials-sg-execution-times"></span><h1>Computation times<a class="headerlink" href="#computation-times" title="Permalink to this headline"></a></h1>
<p><strong>13:12.314</strong> total execution time for <strong>getting-started_tutorials</strong> files:</p> <p><strong>13:16.466</strong> total execution time for <strong>getting-started_tutorials</strong> files:</p>
<table class="docutils align-default"> <table class="docutils align-default">
<colgroup> <colgroup>
<col style="width: 85%" /> <col style="width: 85%" />
@@ -183,23 +183,23 @@
</colgroup> </colgroup>
<tbody> <tbody>
<tr class="row-odd"><td><p><a class="reference internal" href="03-matrix-multiplication.html#sphx-glr-getting-started-tutorials-03-matrix-multiplication-py"><span class="std std-ref">Matrix Multiplication</span></a> (<code class="docutils literal notranslate"><span class="pre">03-matrix-multiplication.py</span></code>)</p></td> <tr class="row-odd"><td><p><a class="reference internal" href="03-matrix-multiplication.html#sphx-glr-getting-started-tutorials-03-matrix-multiplication-py"><span class="std std-ref">Matrix Multiplication</span></a> (<code class="docutils literal notranslate"><span class="pre">03-matrix-multiplication.py</span></code>)</p></td>
<td><p>06:01.523</p></td> <td><p>05:59.250</p></td>
<td><p>0.0 MB</p></td> <td><p>0.0 MB</p></td>
</tr> </tr>
<tr class="row-even"><td><p><a class="reference internal" href="02-fused-softmax.html#sphx-glr-getting-started-tutorials-02-fused-softmax-py"><span class="std std-ref">Fused Softmax</span></a> (<code class="docutils literal notranslate"><span class="pre">02-fused-softmax.py</span></code>)</p></td> <tr class="row-even"><td><p><a class="reference internal" href="02-fused-softmax.html#sphx-glr-getting-started-tutorials-02-fused-softmax-py"><span class="std std-ref">Fused Softmax</span></a> (<code class="docutils literal notranslate"><span class="pre">02-fused-softmax.py</span></code>)</p></td>
<td><p>03:19.799</p></td> <td><p>03:20.262</p></td>
<td><p>0.0 MB</p></td> <td><p>0.0 MB</p></td>
</tr> </tr>
<tr class="row-odd"><td><p><a class="reference internal" href="05-layer-norm.html#sphx-glr-getting-started-tutorials-05-layer-norm-py"><span class="std std-ref">Layer Normalization</span></a> (<code class="docutils literal notranslate"><span class="pre">05-layer-norm.py</span></code>)</p></td> <tr class="row-odd"><td><p><a class="reference internal" href="05-layer-norm.html#sphx-glr-getting-started-tutorials-05-layer-norm-py"><span class="std std-ref">Layer Normalization</span></a> (<code class="docutils literal notranslate"><span class="pre">05-layer-norm.py</span></code>)</p></td>
<td><p>02:12.404</p></td> <td><p>02:12.441</p></td>
<td><p>0.0 MB</p></td> <td><p>0.0 MB</p></td>
</tr> </tr>
<tr class="row-even"><td><p><a class="reference internal" href="01-vector-add.html#sphx-glr-getting-started-tutorials-01-vector-add-py"><span class="std std-ref">Vector Addition</span></a> (<code class="docutils literal notranslate"><span class="pre">01-vector-add.py</span></code>)</p></td> <tr class="row-even"><td><p><a class="reference internal" href="01-vector-add.html#sphx-glr-getting-started-tutorials-01-vector-add-py"><span class="std std-ref">Vector Addition</span></a> (<code class="docutils literal notranslate"><span class="pre">01-vector-add.py</span></code>)</p></td>
<td><p>01:38.111</p></td> <td><p>01:44.030</p></td>
<td><p>0.0 MB</p></td> <td><p>0.0 MB</p></td>
</tr> </tr>
<tr class="row-odd"><td><p><a class="reference internal" href="04-low-memory-dropout.html#sphx-glr-getting-started-tutorials-04-low-memory-dropout-py"><span class="std std-ref">Low-Memory Dropout</span></a> (<code class="docutils literal notranslate"><span class="pre">04-low-memory-dropout.py</span></code>)</p></td> <tr class="row-odd"><td><p><a class="reference internal" href="04-low-memory-dropout.html#sphx-glr-getting-started-tutorials-04-low-memory-dropout-py"><span class="std std-ref">Low-Memory Dropout</span></a> (<code class="docutils literal notranslate"><span class="pre">04-low-memory-dropout.py</span></code>)</p></td>
<td><p>00:00.477</p></td> <td><p>00:00.482</p></td>
<td><p>0.0 MB</p></td> <td><p>0.0 MB</p></td>
</tr> </tr>
</tbody> </tbody>

File diff suppressed because one or more lines are too long

View File

@@ -1,4 +1,4 @@
# Sphinx build info version 1 # Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: e279bcfffe02d90b014352649d6f6a29 config: 4b7bb22acf45d0c0946faffb40b6f21b
tags: 645f666f9bcd5a90fca523b33c5a78b7 tags: 645f666f9bcd5a90fca523b33c5a78b7

Binary file not shown.

Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More