Can confirm that the filter is busted in some way.
I did some informal testing by sending both NSFW text and images to Personal Notes. It didn't trigger on the text or two of the images (which were character reference sheets with very visible dicks), but did on the third image, which had the dick featured much more prominently. It also shouldn't be triggering on Personal Notes or in DMs at all, which points toward a poorly-configured or implemented channel filter, and those features obviously being implemented as channels.
E: Did some perusing of the code, and it appears that they're using an Open Neural Network Exchange model to make an inference on whether an image contains NSFW material on the media proxy level (globally). This is certainly not fine-grained enough to allow it through in some cases but not others; it should live inside the server itself imo
Thread
Comment by @DW01