- Step1: Clone the GitHub repository and install Python dependencies (pip install -r requirements.txt).
- Step2: Define scraping targets by editing the configuration file with URL patterns or search queries.
- Step3: Customize parsing rules and LLM prompt templates to extract and process specific data fields.
- Step4: Provide API credentials for the chosen LLM provider in environment variables.
- Step5: Run the agent script (python agent.py) to start automated scraping and LLM analysis.
- Step6: Review generated summaries, structured JSON outputs, or CSV reports and adjust configurations for further iterations.