Reloading a gradio app while running a function throws errors #8712

irgolic · 2024-07-07T21:52:42Z

Describe the bug

If one is running gradio with reloading (i.e., gradio demo.py instead of python demo.py), runs a function (e.g., presses a "Submit" button), and then changes the file, unexpected behavior occurs.

Specifically:

the app displays Connection errored out. and stops updating the frontend,
the function keeps running,
any time the processing yields a value, it prints a "Returned component not specified as output of function" error.

This happens even if key is set on components, which is counter-intuitive – it feels like it should have enough information to complete the interaction.

From the error I conjecture that the function is still running in the context of the old demo, though the components have been updated in-place to the new demo's, leading to a "Returned component not specified as output of function" error.

The ideal fix would be to preserve execution of the function, and map old output components to new ones under the hood via their key attribute. If the referenced output does not exist anymore, throw a "Returned component not specified as output of function" error as it does now.

Have you searched existing issues? 🔎

I have searched and found no existing issues

Reproduction

import asyncio

import gradio as gr

with gr.Blocks() as demo:
    submit = gr.Button("Submit")

    out = gr.Textbox(key="1")
    outputs = [out]

    # out2 = gr.Textbox(key="2")
    # outputs = [out, out2]

    async def process():
        await asyncio.sleep(5)
        for out_ in outputs:
            yield {out_: 'hi'}

    submit.click(
        process,
        outputs=outputs,
    )

demo.launch()

Create a file with the contents above called script.py
Run gradio script.py
Press the Submit button
Within 5 seconds, uncomment the two lines starting at out2, and save the file

Screenshot

No response

Logs

❯ gradio test.py
Watching: '/Users/rafael/' '/Users/rafael/'

Running on local URL:  http://127.0.0.1:7861

To create a public link, set `share=True` in `launch()`.
Changes detected in: /Users/rafael/script.py
Changes detected in: /Users/rafael/script.py
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/queueing.py", line 541, in process_events
    response = await route_utils.call_process_api(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/route_utils.py", line 276, in call_process_api
    output = await app.get_blocks().process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/blocks.py", line 1938, in process_api
    data = await self.postprocess_data(block_fn, result["prediction"], state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/blocks.py", line 1702, in postprocess_data
    predictions = convert_component_dict_to_list(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/blocks.py", line 638, in convert_component_dict_to_list
    raise ValueError(
ValueError: Returned component <gradio.components.textbox.Textbox object at 0x13a09ab50> not specified as output of function.

System Info

Gradio Environment Information:
------------------------------
Operating System: Darwin
gradio version: 4.37.2
gradio_client version: 1.0.2

------------------------------------------------
gradio dependencies in your environment:

aiofiles: 23.2.1
altair: 5.3.0
fastapi: 0.111.0
ffmpy: 0.3.2
gradio-client==1.0.2 is not installed.
httpx: 0.27.0
huggingface-hub: 0.23.2
importlib-resources: 6.4.0
jinja2: 3.1.4
markupsafe: 2.1.5
matplotlib: 3.9.0
numpy: 1.26.4
orjson: 3.10.6
packaging: 24.0
pandas: 2.2.2
pillow: 10.3.0
pydantic: 2.7.2
pydub: 0.25.1
python-multipart: 0.0.9
pyyaml: 6.0.1
ruff: 0.3.7
semantic-version: 2.10.0
tomlkit==0.12.0 is not installed.
typer: 0.12.3
typing-extensions: 4.12.1
urllib3: 2.2.1
uvicorn: 0.30.1
authlib; extra == 'oauth' is not installed.
itsdangerous; extra == 'oauth' is not installed.


gradio_client dependencies in your environment:

fsspec: 2024.5.0
httpx: 0.27.0
huggingface-hub: 0.23.2
packaging: 24.0
typing-extensions: 4.12.1
websockets: 11.0.3

Severity

I can work around it

The text was updated successfully, but these errors were encountered:

irgolic · 2024-07-07T22:13:47Z

For completeness, when streaming data, the race condition often occurs at a different point.

In the above example, it happens when mapping components to their IDs
https://github.com/gradio-app/gradio/blob/main/gradio/blocks.py#L1671

In the below example, it happens afterward, when trying to dereference the old components' IDs
https://github.com/gradio-app/gradio/blob/main/gradio/blocks.py#L1711

Following the same instructions with this script:

import asyncio

import gradio as gr

with gr.Blocks() as demo:
    submit = gr.Button("Submit")

    out = gr.Textbox(key="1")
    outputs = [out]

    # out2 = gr.Textbox(key="2")
    # outputs = [out, out2]

    async def process():
        for i in range(100000):
            await asyncio.sleep(0.01)
            yield {out: i}

    submit.click(
        process,
        outputs=outputs,
    )

demo.launch()

Commonly produces this error (but can produce the other error too):

❯ gradio test.py
Watching: '/Users/rafael' '/Users/rafael'

Running on local URL:  http://127.0.0.1:7861

To create a public link, set `share=True` in `launch()`.
Changes detected in: /Users/rafael/test.py
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/queueing.py", line 580, in process_events
    response = await route_utils.call_process_api(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/route_utils.py", line 276, in call_process_api
    output = await app.get_blocks().process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/blocks.py", line 1938, in process_api
    data = await self.postprocess_data(block_fn, result["prediction"], state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/blocks.py", line 1759, in postprocess_data
    if block._id in state:
       ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/state_holder.py", line 104, in __contains__
    block = self.blocks_config.blocks[key]
            ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
KeyError: 2

freddyaboulton · 2024-07-09T10:11:53Z

The ideal fix would be to preserve execution of the function, and map old output components to new ones under the hood via their key attribute. If the referenced output does not exist anymore, throw a "Returned component not specified as output of function" error as it does now.

This makes sense. Would you be interested in opening a PR for this @irgolic ?

irgolic · 2024-07-09T10:16:11Z

I can give it a shot.
There should be a lock between the reloading thread and a function's outputs being processed. Any thoughts on where that should be managed?

irgolic added the bug Something isn't working label Jul 7, 2024

irgolic linked a pull request Jul 10, 2024 that will close this issue

[RFC] Reload while running fn #8738

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reloading a gradio app while running a function throws errors #8712

Reloading a gradio app while running a function throws errors #8712

irgolic commented Jul 7, 2024

irgolic commented Jul 7, 2024

freddyaboulton commented Jul 9, 2024

irgolic commented Jul 9, 2024

Reloading a gradio app while running a function throws errors #8712

Reloading a gradio app while running a function throws errors #8712

Comments

irgolic commented Jul 7, 2024

Describe the bug

Have you searched existing issues? 🔎

Reproduction

Screenshot

Logs

System Info

Severity

irgolic commented Jul 7, 2024

freddyaboulton commented Jul 9, 2024

irgolic commented Jul 9, 2024