Software Engineer - Python | Remote

Position: Senior Software Engineer – Python (LLM Evaluation \& Repository Validation) Type: Contractor Assignment (3 months) Compensation: $19-$20/hr Location: Remote Commitment: 40 hrs/week with some overlap with PST Role Responsibilities * Evaluate and analyze GitHub issues across widely used open-source repositories. * Set up and configure development environments for repositories, including Docker setup. * Modify and run codebases locally to evaluate LLM performance in debugging and bug-fixing scenarios. * Evaluate unit test coverage and assess the quality of existing tests. * Work with real-world Python repositories to identify engineering tasks suitable for LLM evaluation. * Collaborate with researchers to expand datasets used for training and evaluating AI models. * Identify repositories and issues that present challenging real-world scenarios for LLMs. * Support the creation of realistic software engineering tasks based on repository histories. Requirements * Strong experience in software development. * Strong experience with Python programming. * Proficiency in Git, Docker, and development environment setup. * Experience working with well-maintained public GitHub repositories. * Ability to understand, modify, and test complex codebases. * Strong analytical thinking and problem-solving skills. * Excellent written English and ability to document findings clearly. * Ability to work independently in a remote environment. * Desktop or laptop with a stable internet connection. Preferred Qualifications * Experience contributing to open-source projects. * Exposure to LLM evaluation, AI research, or machine learning projects. * Experience with developer tools, automation agents, or CI/CD pipelines. Application Process * Application Form * ICF + Profile Review * Assessment * Submit