OpenAI

Disclaimer: I worked on this project independently during personal time. Nothing here represents the views or endorsement of SGS. Any opinions, findings, and conclusions expressed in this blog post are solely mine. The project utilizes Python, OpenAI’s language models and STARLIMS mock code I created, which may have separate terms of use and licensing agreements.

AI is not a magician; it’s a tool. It is a tool for amplifying innovation

Fei-Fei Li

With this in mind, imagine: what if we could automatically get STARLIMS Code Review feedback? You know, an extra quality layer powered by AI?

“STARLIMS is a proprietary language” you will say.

“It has to be trained” you will say.

True; yet, what if?…

I have done another experiment. I was given a challenge to try Python and OpenAI API, but I wasn’t really given any background. Given my recent fun with CI/CD and the fact I’m back working on the STARLIMS product, I thought “Can we automatically analyze STARLIMS Code? Like an automated code-reviewer?” Well, yes, we can!

As I was recommended a long time ago, with this project, let me show you the end result. I have a REST API running locally (Python Flask) with the following 2 endpoints:

POST /analyze/<language>

POST /analyze/<language>/<session_id>

The 1st will kick a new analysis session, and the 2nd allows the user to continue their analysis session (like queuing scripts and relating them together!!!)

I usually create nice diagrams, but for this, really, the idea is

STARLIMS <-> Python <-> Open AI

So no diagram for you today! How does it work?

I can pass SSL code to a REST API, and receive this:

{
    "analysis": {
        "feedback": [
            {
                "explanation": "Defaulting a parameter with a numeric value may lead to potential issues if the parameter is expected to be a string. It's safer to default to 'NIL' or an empty string when dealing with non-numeric parameters.",
                "snippet": ":DEFAULT nItemId, 1234;",
                "start_line": 4,
                "suggestion": "Consider defaulting 'nItemId' to 'NIL' or an empty string depending on the expected data type.",
                "type": "Optimization"
            }
        ]
    },
    "session_id": "aa4b3bd3-75bd-42e3-8f31-e53502e68256"
}

It works with STARLIMS Scripting Language (SSL), STARLIMS Data sources (DS) and … JScript! Here’s an example of a JScript output:

{
    "analysis": {
        "items": [
            {
                "detailed explanation": "Checking for an empty string using the comparison operator '==' is correct, but using 'Trim()' method before checking can eliminate leading and trailing white spaces.",
                "feedback type": "Optimization",
                "snippet of code": "if (strMaterialType == '')",
                "start line number": 47,
                "suggestion": "Update the condition to check for an empty string after trimming: if (strMaterialType.trim() === '')"
            },
            {
                "detailed explanation": "Using the logical NOT operator '!' to check if 'addmattypelok' is false is correct. However, for better readability and to avoid potential issues, it is recommended to explicitly compare with 'false'.",
                "feedback type": "Optimization",
                "snippet of code": "if (!addmattypelok)",
                "start line number": 51,
                "suggestion": "Update the condition to compare with 'false': if (addmattypelok === false)"
            },
            {
                "detailed explanation": "Checking the focused element is a good practice. However, using 'Focused' property directly can lead to potential issues if the property is not correctly handled in certain scenarios.",
                "feedback type": "Optimization",
                "snippet of code": "if ( btnCancel.Focused )",
                "start line number": 58,
                "suggestion": "Add a check to ensure 'btnCancel' is not null before checking its 'Focused' property."
            }
        ]
    },
    "session_id": "7e111d84-d6f4-4ab0-8dd6-f96022c76cff"
}

How cool is that? To achieve this, I used Python and OpenAI API. I had to purchase some credits; but really, it is cheap enough and worth it when used to a small scale (like a development team). I put 10$ in there, and I have been running many tests (maybe a few hundreds) and I am down by 0.06$, so… I would say worth it.

The beauty of this is my project supports this:

• Add new languages in 5 minutes (just need to add the class, update the prompt, add the reference code, restart the app, go!)
• Enhance accuracy by providing good code, training the assistant what is valid code

To give you an idea, the project is very small:

Looking ahead with this small project, I’m thinking beyond just checking code for errors. Imagine if we could hook it up to our DevOps setup, like Azure DevOps or SonarQube. It would be like having a digital assistant that not only spots issues but also files bugs and suggests improvements automatically! This means smoother teamwork, better software quality, and fewer headaches for all of us.

Now that I got this working, I am thinking about bunch of exciting ideas like:

• Integrate this as a Quality Gate on commits.
  • If it fails, goes back to developer
  • If it succeeds, record the results and run the pull request (or push to the next human reviewer)
• Implement a mechanism for automatic Unit Tests generation (we potentially can do something there!)
• Implement a mechanism for code coverage report (also possible!)
• Integration of these to STARLIMS directly so we can benefit from this and include in a CI/CD pipeline somehow

Dreaming is free, is it not? Well, not quite in this case, but I’m good for another 9.94$…

I have the repo set as private on Github. This is a POC, but I think it can be a very cool thing for STARLIMS, but also will work for any other proprietary language if I get some good sample code.

Hell, it can even work for already supported languages like Javascript, c#, or anything, without training! So we could use this pattern for virtually any code review.

Interested? Leave a comment or shoot me an email!