Authors
Minim dolor ad ullamco excepteur aute aliquip reprehenderit ipsum cillum. Labore cupidatat aliquip enim sit do occaecat quis in occaecat exercitation. Occaecat culpa laboris occaecat dolore adipisicing.
Minim dolor ad ullamco excepteur aute aliquip reprehenderit ipsum cillum. Labore cupidatat aliquip enim sit do occaecat quis in occaecat exercitation. Occaecat culpa laboris occaecat dolore adipisicing. Culpa ullamco reprehenderit reprehenderit exercitation fugiat excepteur exercitation cillum fugiat nisi voluptate. Irure labore fugiat veniam minim est in proident eiusmod est irure.
new ebook form
This form requires targeting cookies to load.
To view this form, please accept targeting cookies.
Cras mattis consectetur purus sit amet fermentum. Maecenas faucibus mollis interdum. Morbi leo risus, porta ac consectetur ac, vestibulum at eros. Duis mollis, est non commodo luctus, nisi erat porttitor ligula, eget lacinia odio sem nec elit.
Aenean lacinia bibendum nulla sed consectetur. Maecenas faucibus mollis interdum. Nullam quis risus eget urna mollis ornare vel eu leo. Aenean eu leo quam. Pellentesque ornare sem lacinia quam venenatis vestibulum.
Occaecat culpa laboris occaecat dolore adipisicing. Culpa ullamco reprehenderit reprehenderit exercitation fugiat excepteur exercitation cillum fugiat nisi voluptate. Irure labore fugiat veniam minim est in proident eiusmod est irure.
Occaecat culpa laboris occaecat dolore adipisicing. Culpa ullamco reprehenderit reprehenderit exercitation fugiat excepteur exercitation cillum fugiat nisi voluptate. Irure labore fugiat veniam minim est in proident eiusmod est irure.
Occaecat culpa laboris occaecat dolore adipisicing. Culpa ullamco reprehenderit reprehenderit exercitation fugiat excepteur exercitation cillum fugiat nisi voluptate. Irure labore fugiat veniam minim est in proident eiusmod est irure.
Use cases
Unstructured
uam eget aliquam dolor. In nisl massa, egestas sit amet nulla sed, bibendum porttitor orci. Sed in venenatis sem. Vestibulum suscipit augue vitae augue semper, vitae mollis felis posuere.
an unordered list:
- Toothbrush
- Toothpaste
- Dental Floss
- Sub Something
- And another one
- hmm
- Soap
- Deodorant
- Shampoo
- Conditioner
an ordered list:
- Toothbrush
- test
- asdf
- Toothpaste
- Dental Floss
- Soap
- Deodorant
- Shampoo
- Conditioner
- we
- need
- more
- items
- to
- test
Unstructured Data Dominance
Most enterprise knowledge, aka the exact context your LLM-based application requires, lives in unstructured formats - PDFs, Word documents, PowerPoint presentations, internal wiki pages, emails, and chat messages. Traditional ETL tools weren't designed to handle these formats effectively. They are not built for unstructured data.
Handling a plethora of file formats
Each file type - whether it's a PDF, DOCX, PPT, or some other format - has its own internal structure and encoding. You need different parsers and extraction strategies for each format, significantly increasing complexity and maintenance overhead.
Unstructured Platform addresses this challenge by providing comprehensive data transformation capabilities that can handle over 60 different types of unstructured data formats. The platform employs a multi-layered approach: rule-based parsers efficiently handle structured documents like HTML, markdown, and Word files; custom fine-tuned OCR and document understanding models process image-based content from PDFs and PowerPoint presentations; and for the most challenging documents like noisy scans, handwritten notes, and complex tables and forms, the platform leverages state-of-the-art models such as, for example, Claude Sonnet and GPT-4o. The combination of parsing strategies ensures reliable content extraction across a wide range of document types while remaining cost-effective. Unstructured intelligently routes documents through the appropriate strategy, optimizing both processing speed and cost.
- Native File Access via Volumes
Mount and stream files from Databricks Volumes without third-party connectors. Supports 60+ formats with OCR, VLM, and parsing capabilities baked in - GenAI-Optimized Data Transformation
Mount and stream files from Databricks Volumes without third-party connectors. Supports 60+ formats with OCR, VLM, and parsing capabilities baked in - Delta Table Integration
Mount and stream files from Databricks Volumes without third-party connectors. Supports 60+ formats with OCR, VLM, and parsing capabilities baked in
Each connector requires credentials via .env, with full setup instructions available here.
This is a caption.
Unstructured API offers rich functionality with support for many different connectors and actions to manage them, as well as functionality to manage the workflows.
Unstructured
The Tool Count Matters: Less Can Be More
Unstructured API offers rich functionality with support for many different connectors and actions to manage them, as well as functionality to manage the workflows. We quickly realized that matching API functionality to MCP tools one to one would not be the best MCP design. Too many tools available can overwhelm and confuse LLMs, making it harder for the model to find precise tools for the tasks at hand. More importantly, having an excessive number of tools creates a documentation challenge, as it quickly consumes the available context space that the LLM can use. This is why it's critical to find a balance in the number of tools available.
To ease the context window management challenge, we abstracted all of the connector management functionality. This resulted in a slashing of the context window usage by 5000 tokens!
Slider Test
This content is hosted by YouTube.
To view it, please accept targeting cookies.
Copy copy
While the Unstructured MCP server supports many different tools, the reality is that not all tools are needed in every situation. Depending on your use case, you can restrict the MCP server to a smaller, more targeted set of tools. This will reduce the token usage for the documentation, may improve performance, and make it easier for LLMs to understand and execute their tasks.
import os
from dataclasses import dataclass
from pydantic_ai import Agent
from pydantic_ai.mcp import MCPServerHTTP
from pydantic_ai.tools import ToolDefinition
@dataclass
class MCPServerHTTPWithSubsetTools(MCPServerHTTP):
selected_tools: list[str]
async def list_tools(self) -> list[ToolDefinition]:
all_available_tools = await super().list_tools()
filtered_tools = [
tool for tool in all_available_tools if tool.name in self.selected_tools
]
return filtered_tools
s3_tools = MCPServerHTTPWithSubsetTools(
url=os.getenv("UNSTRUCTURED_MCP_SERVER_URL"),
selected_tools=[
"create_s3_source",
"update_s3_source",
"delete_s3_source",
],
)
source_configuration_agent = Agent(
system_prompt="Your goals is to configure S3 source in Unstructured platform",
mcp_servers=[s3_tools],
)
Caption
Another Caption
