100 Days of Gen AI: Day 32

I got the AI guardrail filter working, kinda. The web UI is connected to a python server which can interact with my guardrail policy in AWS. All the UI and server plumbing works, but this guardrail feature leaves something to be desired. From this screenshot you can see it only catches two of the four leaked types of information:

Then in the AWS test editor, on a slightly different query you can see it triggers a false positive on bank account info, and misses three other types of sensitive info.

It seems setting up robust guardrails is a bit more involved than just using this AWS feature out of the box.