⚠️ Security Advisory

IMPORTANT: The element feature currently contains a known security vulnerability that could allow unauthorized file access. We strongly recommend against using elements in production environments until a comprehensive fix is implemented in an upcoming release.

Key Improvements

• Complete overhaul with OpenAI Realtime API support for streaming audio chat interactions by @willydouhard (#1401, #1406, #1410)
• New interactive DataFrame display functionality with auto-fit content by @desertproject and @hayescode (#1373, #1467)
• Enhanced security measures and development tooling by @dokterbob (#1431, #1414)

Breaking Changes

OpenAI Realtime API Integration

• Replaced AudioChunk with InputAudioChunk and OutputAudioChunk
• Changed default audio sampling rate from 44100 to 24000
• Removed several audio configuration options (min_decibels, initial_silence_timeout, silence_timeout, chunk_duration, max_duration)
• Removed RecordScreen component

Other Changes

New Features

• Implemented realtime audio streaming with new components by @willydouhard (#1401, #1406, #1410):
  • Added AudioPresence for visual feedback
  • Introduced WavRecorder and WavStreamPlayer classes
  • Added audio interruption functionality
  • New on_audio_start callback
• Added interactive DataFrame display using MUI Data Grid with auto-fit content by @desertproject and @hayescode (#1373, #1467)
• Enhanced image interaction with popup view and download capabilities by @fgalind1 (#1402)
• Made websocket connections optional in react-client by @sandangel (#1379)
• Added current URL to message payload in copilot mode by @fgalind1 (#1403)
• Enabled empty chat input when submitting attachments by @EcoleKeine (#1261)
• Added support for regional language variants like es-419 by @erauld (#1399)

Technical Improvements

• Factored storage clients into separate modules by @ndricca (#1363)
• Implemented comprehensive linting with ruff by @dokterbob (#1495)
• Added mypy daemon for faster type-checking by @dokterbob (#1495)
• Enhanced GitHub Actions with additional linting by @dokterbob (#1445)
• Enabled direct installation from GitHub by @dokterbob (#1423)
• Various build script improvements by @dokterbob (#1462)

Migration Guide

OpenAI Realtime API Migration

If you're using audio features, you'll need to update your code to use the new realtime audio system:

1. Update imports and types:

- from chainlit.types import AudioChunk
+ from chainlit.types import InputAudioChunk, OutputAudioChunk

2. Update your audio callbacks:

@cl.on_audio_start
async def on_audio_start():
    # New callback to initialize audio session
    # Return True to enable audio connection
    return True

@cl.on_audio_chunk
async def on_audio_chunk(chunk: cl.InputAudioChunk):
    # Process incoming audio chunks
    # chunk.data contains the raw audio data
    pass

@cl.on_audio_end
async def on_audio_end():
    # Clean up audio session
    pass

3. For streaming audio back to the client:

await cl.context.emitter.send_audio_chunk(
    cl.OutputAudioChunk(
        mimeType="pcm16",
        data=audio_data,
        track=track_id
    )
)

See our documentation for a complete implementation example.

New Contributors

• @fgalind1 made their first contribution with URL and image interaction improvements (#1403)
• @erauld made their first contribution with regional language support (#1399)
• @ndricca made their first contribution with storage client modularization (#1363)
• @desertproject made their first contribution with interactive DataFrame display (#1373)
• @EcoleKeine made their first contribution with attachments handling improvements (#1261)
• @sandangel made their first contribution with optional websocket connections (#1379)

Full Changelog: https://github.com/Chainlit/chainlit/compare/1.3.1...2.0rc0