Answer
A useful way to structure this is to bring the vendor a real scenario from the environment instead of letting them run their own polished script. Describe an actual multi-step process, such as a nightly file transfer feeding a Db2 for i update that triggers a downstream EDI transmission, and ask the vendor to build it live or walk through exactly how their tool would handle each step, including what happens if the file arrives an hour late or arrives with a data format it has not seen before. Watching a vendor think through an unfamiliar scenario in real time reveals far more about the product than a rehearsed demo ever will.
Red flags worth watching for include a vendor who cannot explain how failed jobs get retried without manual intervention, who has no clear answer for how an admin would investigate a failure from six weeks ago, or who treats alerting as an afterthought rather than a core feature. It is also worth asking who gets notified when something fails, whether that notification includes enough context to act on immediately, and whether the audit trail would satisfy an internal auditor asking why a particular job ran at 2am with elevated authority. A vendor that welcomes these questions is usually more confident in their product than one who tries to redirect back to the schedule-building demo.