I was at a political science seminar recently, where every participant had read (and was expected to have read) the paper beforehand. The speaker was given 10 minutes (no interruptions) to make a brief statement, maybe regarding what they hoped to get out of the seminar that day, or what they thought some of the limitations and next steps would be.
People then put up their hands if they had comments. Fifteen hands shot up at once, and a list of names was recorded, which was followed in sequence, with a few additional names added to the list as the seminar went on. It was a remarkably civilized experience, and proved that it is possible to have a large amount of audience participation and back-and-forth discussion, without derailing the speaker or yielding the entire seminar to one or two intransigent audience members.
It might be hard to do this for a job talk, where there is a certain set of ritualistic expectations, but it seems like for regular academic seminars, this is a model that might be worth exploring.