added info smart_open module (#25000)
This commit is contained in:
committed by
Paul Gamble
parent
dc9839d908
commit
ce212d6c35
@ -0,0 +1,31 @@
|
||||
---
|
||||
title: smart_open 1.7.1
|
||||
---
|
||||
|
||||
## Python smart_open 1.7.1
|
||||
|
||||
smart_open is a python library for efficient streaming of large files from/to HDFS, S3, HTTP, or local files.
|
||||
The simplest way to install `smart_open` is using `pip`
|
||||
|
||||
``` pip install smart_open ```
|
||||
|
||||
It is a good replacement for built-in open().
|
||||
It does everything open() can, and falls back to native open whenever possible.
|
||||
|
||||
#### Why?
|
||||
|
||||
It can be a pain while working with large S3 files using boto and boto3 libraries.
|
||||
`smart_open` builds on boto3 and is very simple to work with.
|
||||
|
||||
|
||||
``` python
|
||||
import smart_open
|
||||
|
||||
with smart_open.smart_open('s3://tmp-bucket/myfile.txt') as f:
|
||||
lines = f.readlines()
|
||||
for line in lines:
|
||||
print line
|
||||
|
||||
```
|
||||
|
||||
`smart_open` is an open source software, maintained on [GitHub](https://github.com/RaRe-Technologies/smart_open)
|
Reference in New Issue
Block a user