Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Filter by Categories
About Article
Analyze Data
Archive
Best Practices
Better Outputs
Blog
Code Optimization
Code Quality
Command Line
Course
Daily tips
Dashboard
Data Analysis & Manipulation
Data Engineer
Data Visualization
DataFrame
Delta Lake
DevOps
DuckDB
Environment Management
Feature Engineer
Git
Jupyter Notebook
LLM
LLM Tools
Machine Learning
Machine Learning & AI
Machine Learning Tools
Manage Data
MLOps
Natural Language Processing
Newsletter Archive
NumPy
Pandas
Polars
PySpark
Python Helpers
Python Tips
Python Utilities
Scrape Data
SQL
Testing
Time Series
Tools
Visualization
Visualization & Reporting
Workflow & Automation
Workflow Automation

Newsletter #287: Granite Vision: Convert Chart Images to CSV with Transformers

Newsletter #287: Granite Vision: Convert Chart Images to CSV with Transformers

Grab your coffee. Here are this week’s highlights.


📅 Today’s Picks

Convert Chart Images to CSV with IBM Granite Vision

Code example: Convert Chart Images to CSV with IBM Granite Vision

Problem

Chart data often contains valuable insights, but extracting numbers from these charts manually is time-consuming and tedious.

Solution

IBM’s Granite Vision 3.3 2B converts chart images directly into structured CSV data using Hugging Face Transformers.

Here’s how to extract structured data from any chart image in three steps.

1. Load the Model

Load the chart-to-CSV model from HuggingFace using the transformers library.

<span class="kn">from</span><span class="w"> </span><span class="nn">transformers</span><span class="w"> </span><span class="kn">import</span> <span class="n">AutoProcessor</span><span class="p">,</span> <span class="n">AutoModelForVision2Seq</span>
<span class="kn">from</span><span class="w"> </span><span class="nn">huggingface_hub</span><span class="w"> </span><span class="kn">import</span> <span class="n">hf_hub_download</span>
<span class="kn">from</span><span class="w"> </span><span class="nn">PIL</span><span class="w"> </span><span class="kn">import</span> <span class="n">Image</span>
<span class="kn">import</span><span class="w"> </span><span class="nn">torch</span>

<span class="n">model_path</span> <span class="o">=</span> <span class="s2">"ibm-granite/granite-vision-3.3-2b-chart2csv-preview"</span>
<span class="n">device</span> <span class="o">=</span> <span class="s2">"cuda"</span> <span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">cuda</span><span class="o">.</span><span class="n">is_available</span><span class="p">()</span> <span class="k">else</span> <span class="s2">"cpu"</span>

<span class="n">processor</span> <span class="o">=</span> <span class="n">AutoProcessor</span><span class="o">.</span><span class="n">from_pretrained</span><span class="p">(</span><span class="n">model_path</span><span class="p">)</span>
<span class="n">model</span> <span class="o">=</span> <span class="n">AutoModelForVision2Seq</span><span class="o">.</span><span class="n">from_pretrained</span><span class="p">(</span><span class="n">model_path</span><span class="p">)</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">)</span>
2. Prepare Your Chart

Define the chart image and task instruction in a conversation format.

Chart Example

<span class="c1"># Load a chart image</span>
<span class="n">img_path</span> <span class="o">=</span> <span class="n">hf_hub_download</span><span class="p">(</span>
    <span class="n">repo_id</span><span class="o">=</span><span class="n">model_path</span><span class="p">,</span> <span class="n">filename</span><span class="o">=</span><span class="s2">"example.jpg"</span>
<span class="p">)</span>
<span class="n">img</span> <span class="o">=</span> <span class="n">Image</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="n">img_path</span><span class="p">)</span>

<span class="c1"># Use the chart-to-CSV prompt</span>
<span class="n">conversation</span> <span class="o">=</span> <span class="p">[</span>
    <span class="p">{</span><span class="s2">"role"</span><span class="p">:</span> <span class="s2">"user"</span><span class="p">,</span> <span class="s2">"content"</span><span class="p">:</span> <span class="p">[</span>
        <span class="p">{</span><span class="s2">"type"</span><span class="p">:</span> <span class="s2">"image"</span><span class="p">,</span> <span class="s2">"url"</span><span class="p">:</span> <span class="n">img_path</span><span class="p">},</span>
        <span class="p">{</span><span class="s2">"type"</span><span class="p">:</span> <span class="s2">"text"</span><span class="p">,</span> <span class="s2">"text"</span><span class="p">:</span> <span class="s2">"Parse the chart in the image to CSV format."</span><span class="p">}</span>
    <span class="p">]}</span>
<span class="p">]</span>
3. Generate CSV Output

Apply the chat template, generate tokens, and decode back to CSV text.

<span class="n">inputs</span> <span class="o">=</span> <span class="n">processor</span><span class="o">.</span><span class="n">apply_chat_template</span><span class="p">(</span>
    <span class="n">conversation</span><span class="p">,</span>
    <span class="n">add_generation_prompt</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
    <span class="n">tokenize</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
    <span class="n">return_dict</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
    <span class="n">return_tensors</span><span class="o">=</span><span class="s2">"pt"</span>
<span class="p">)</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">device</span><span class="p">)</span>

<span class="n">output</span> <span class="o">=</span> <span class="n">model</span><span class="o">.</span><span class="n">generate</span><span class="p">(</span><span class="o">**</span><span class="n">inputs</span><span class="p">,</span> <span class="n">max_new_tokens</span><span class="o">=</span><span class="mi">500</span><span class="p">)</span>
<span class="n">csv_output</span> <span class="o">=</span> <span class="n">processor</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="n">output</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">skip_special_tokens</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">csv_output</span><span class="p">)</span>

Output:

State,2017,2018
NJ,4.6,4.1
CT,4.7,4.1
DE,4.5,3.8
NY,4.7,4.1
PA,4.9,4.3

PydanticAI: Control OpenAI Data Retention with openai_store

Code example: PydanticAI: Control OpenAI Data Retention with openai_store

Problem

By default, OpenAI may retain your API request data for internal review and model improvement. For healthcare, finance, and legal applications, this default creates compliance risks you can’t afford.

Solution

PydanticAI v1.52.0 introduces the openai_store setting to explicitly disable data retention in one line.


☕️ Weekly Finds

msgvault [Data Management] – Archive a lifetime of email and chat locally with full Gmail backup, search, DuckDB-powered analytics, an interactive TUI, and an MCP server for querying messages with AI

monty [Developer Tools] – Minimal, secure Python interpreter written in Rust designed for use by AI agents, providing sandboxed code execution with safety guarantees

baserow [No-Code Platform] – Open-source no-code platform for building databases, applications, automations, and AI agents with enterprise-grade security and self-hosted deployment options

Looking for a specific tool? Explore 70+ Python tools →

Stay Current with CodeCut

Actionable Python tips, curated for busy data pros. Skim in under 2 minutes, three times a week.

Leave a Comment

Your email address will not be published. Required fields are marked *

0
    0
    Your Cart
    Your cart is empty
    Scroll to Top

    Work with Khuyen Tran

    Work with Khuyen Tran