FastAPI UploadFile慢速

问题描述

您好，我对FastAPI上载文件速度有问题。

如果我如下所示创建和结束：

@app.post("/report/upload")
def create_upload_files(files: UploadFile = File(...)):
        try:
            with open(files.filename,'wb+') as wf:
                wf.write(file.file.read())
                wf.close()
        except Exception as e:
            return {"error": e.__str__()}

使用uvicorn：

../venv/bin/uvicorn test_upload:app --host=0.0.0.0 --port=5000 --reload

我正在做一些测试，根据请求上载一个大约100M的文件，大约需要128秒。

f = open(sys.argv[1],"rb").read()
hex_convert = binascii.hexlify(f)
items = {"files": hex_convert.decode()}
start = time.time()
r = requests.post("http://192.168.0.90:5000/report/upload",files=items)
end = time.time() - start
print(end)

我测试了在flask中编写API端点的相同上传脚本，花费了大约0.5秒

from flask import Flask, render_template, request
app = Flask(__name__)


@app.route('/uploader', methods = ['GET', 'POST'])
def upload_file():
   if request.method == 'POST':
      f = request.files['file']
      f.save(f.filename)
      return 'file uploaded successfully'

if __name__ == '__main__':
    app.run(host="192.168.0.90",port=9000)

我是否做错了什么？

感谢您的帮助

推荐答案

终结点应为异步，并且由于所有UploadFilemethods are async methods, you need to "await" them。因此，文件内容应读作：contents = await myfile.read()。

您可以使用同步写入方式写入文件，如下所示：

@app.post("/upload")
async def upload(files: List[UploadFile] = File(...)):
    for file in files:
        with open(file.filename, 'wb') as f:
            contents = await file.read()
            f.write(contents)
            
    return {"Uploaded Filenames": [file.filename for file in files]}

或(更好)使用aiofiles的异步写入：

@app.post("/upload")
async def upload(files: List[UploadFile] = File(...)):
    for file in files:
        async with aiofiles.open(file.filename, 'wb') as f:
            contents = await file.read()
            await f.write(contents)
            
    return {"Uploaded Filenames": [file.filename for file in files]}

或异步in the chunked manner, to avoid loading the entire file into memory。不过，此操作需要更长的时间才能完成(取决于您选择的区块大小)。

@app.post("/upload")
async def upload(files: List[UploadFile] = File(...)):
    for file in files:
        async with aiofiles.open(file.filename, 'wb') as f:
            while content := await file.read(1024): # async read chunk
                await f.write(content)
            
    return {"Uploaded Filenames": [file.filename for file in files]}

通过Python请求测试代码：

url = 'http://127.0.0.1:8000/upload'
files = [('files', open('test_files/a.txt', 'rb')), ('files', open('test_files/b.txt', 'rb'))]
resp = requests.post(url=url, files = files) 
print(resp.json())