Skip to content

locchh/jsonparser

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

JSON Parser

A comprehensive framework for applying AI models to generate structured JSON.

Features

  • Multiple Parser Implementations:

    • GPTParser: Uses OpenAI API to generate structured JSON
    • JSONParser: Uses open-source transformer models
    • RecursiveRetryParser: Implements a recursive retry pattern with exponential backoff
    • OpenAIParserBeta: Leverages OpenAI's beta API with built-in JSON schema support
  • Analysis Tools:

    • Compliance Analysis: Measure how well JSON instances comply with JSON schemas
    • Complexity Analysis: Evaluate the structural complexity of JSON outputs
    • Consistency Analysis: Compare cross-parser agreement and self-consistency
    • Correctness Analysis: Measure how closely parser outputs match ground-truth structures
  • Utility Functions:

    • JSON schema validation
    • OpenAI schema compatibility checking
    • JSON comparison with similarity scoring
    • Visualization tools for analysis results

Installation

# Clone the repository
git clone https://github.com/locchh/jsonparser.git
cd jsonparser

# Install the package
pip install -e .

Quick Start

from jsonparser.parsers import GPTParser
from jsonparser.utils.helper import calculate_json_complexity

# Initialize a parser
parser = GPTParser()

# Define a schema
schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "age": {"type": "integer"}
    },
    "required": ["name"]
}

# Parse content
content = "John Doe is 30 years old."
result = parser(content, schema)

# Analyze complexity
complexity = calculate_json_complexity(result)
print(f"JSON Complexity: {complexity}")

Command-Line Tools

The package provides several command-line tools:

# Validate a JSON instance against a schema
validate-json instance.json schema.json

# Validate a JSON schema
validate-schema schema.json

# Validate a schema for OpenAI compatibility
validate-openai-schema schema.json

# Compare two JSON files
compare-json file1.json file2.json

Project Structure

jsonparser/
├── .gitignore
├── LICENSE (MIT)
├── README.md
├── examples/
│   └── quickstart.py
├── requirements.txt
├── schemas/
│   ├── book.json
│   ├── car.json
│   └── [8 more schema files]
├── setup.py
└── src/
    └── jsonparser/
        ├── __init__.py
        ├── errors/
        │   ├── __init__.py
        │   ├── error_handling.py
        │   └── error_types.py
        ├── logging/
        │   ├── __init__.py
        │   └── logging_config.py
        ├── parsers/
        │   ├── __init__.py
        │   └── parser.py
        └── utils/
            ├── __init__.py
            ├── compare_json_instance.py
            ├── helper.py
            ├── validate_json_instance.py
            ├── validate_json_schema.py
            └── validate_openai_schema.py

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgements

This project was inspired by jsonformer, which demonstrates an elegant approach to structured JSON generation using language models. The jsonformer project has been a significant influence and passion that motivated the development of this framework.

References

About

A comprehensive framework for applying AI models to generate structured JSON.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages