Advancing Explainable AI in Radiology Research with NVIDIA Clara Reason

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from PIL import Image


# Load the Clara Reason model 
model_name = "nvidia/NV-Reason-CXR-3B"
model = AutoModelForImageTextToText.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
).eval().to("cuda")


processor = AutoProcessor.from_pretrained(model_name)

# Load chest x-ray image
image = Image.open("chest_xray.png")

# Prepare input with clinical context
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": image,
            },
            {
                "type": "text",
                "text": "Find abnormalities and support devices."
            }
        ]
    }
]

# Create prompt using chat template
text = processor.apply_chat_template(
    messages,
    add_generation_prompt=True
)

# Process inputs
inputs = processor(
    text=text,
    images=[image],
    return_tensors="pt"
).to(model.device)

# Generate reasoning with chain-of-thought
generated_ids = model.generate(
    **inputs,
    max_new_tokens=2048
)

# Trim the generated ids to remove the input ids
trimmed_generated_ids = [
    out_ids[len(in_ids):]
    for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]


# Decode the response
generated_text = processor.batch_decode(
    trimmed_generated_ids,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False
)[0]


print("Output:")
print(generated_text)

THINK: We'll begin with the standard assessment of this AP chest x-ray. As you possibly can see, the lung fields are relatively well visualized, although the patient's position and the indisputable fact that it's a conveyable AP view may limit the evaluation to some extent. I feel the standard is suitable for such a examination.

Now, we're taking a look at the medical devices and instrumentation. I do not see any medical devices equivalent to central venous catheters, endotracheal tubes, or chest tubes. Nonetheless, I do notice that the patient is probably going in a clinical setting, given the portable nature of the examination.

The central airways appear patent. The trachea is visible and never deviated, and the proper and left most important bronchi appear unremarkable. I do not see any foreign objects or abnormalities within the central airways.

Moving on to the proper lung, we're evaluating the upper lobe first. The aeration of the proper upper lobe appears reduced, which might be suggestive of some volume loss or atelectasis. I do not see any distinct masses or nodules, but there are some subtle opacities that would represent infiltrates or edema.

Next, we are going to evaluate the proper middle and lower lobes. The lung volumes appear low, which might be indicative of some atelectasis or volume loss. I see some patchy opacities in the proper lower lobe, which can represent pulmonary edema or infiltrates. The costophrenic angle is barely blunted, which might be significant for a small pleural effusion.

Now, let's examine the left lung. The left upper lobe appears just like the proper, with reduced aeration and a few subtle opacities that would represent infiltrates or edema. The left lower lobe also shows some patchy opacities and a rather blunted costophrenic angle, suggestive of a small pleural effusion.

We're taking a look at the pleura now, and as I discussed before, the costophrenic angles are barely blunted bilaterally, which is concerning for small bilateral pleural effusions. I do not see any evidence of pneumothorax.

The mediastinum appears enlarged, which might be indicative of cardiomegaly. The cardiac silhouette is outstanding, and I feel this finding could also be meaningful for moderate cardiomegaly.

The hila appear unremarkable, with none significant adenopathy or masses. The lung vasculature appears outstanding, which might be suggestive of pulmonary edema.

The center, as I discussed before, appears enlarged, with a cardiothoracic ratio that's elevated. This finding is concerning for cardiomegaly.

Now, we're taking a look at the upper abdomen, and I see some normal-appearing bowel gas patterns. There isn't a free air under the diaphragm, which is very important to notice.

Finally, we're taking a look at the bony structures. The clavicles, scapulae, and ribs appear unremarkable, with none focal lesions or fractures. The vertebral bodies are well visualized and appear normal.

In summary, probably the most significant findings on this patient are the presence of mild pulmonary edema, small bilateral pleural effusions, and moderate cardiomegaly. These findings could represent a spread of clinical conditions, including heart failure or other cardiopulmonary diseases. 

ANSWER: Cardiomegaly, Edema, Enlarged Cardiomediastinum, Lung Opacity, Pleural Effusion