I “KID YOU NOT” – Catchphrase used by Jack Paar. Paar, host of the Tonight Show from 1957 to 1962, ‘invented the talk-show format as we know it: the ability to sit down and make small talk big,’ said Merv Griffin. ‘Even youngsters sent to bed before Mr. Paar came on parroted his jaunty catchphrase, ‘I kid you not.’ From “He invented late-night talk, then walked away,” an article in the Herald-Leader, Lexington, Ky., January 28, 2004.
Grammatically both your versions are correct – “kid you” and “kid with you”
Your wording of the question suggests someone is upset about a joke you’ve played on them, so it’s more common to say
“Just kidding” rather than pose it as a question.
Herman Wouk’s Cain Mutiny
It’s quasi-archaic inversion, combined with the informal “kid” draws attention to the fact that the speaker is being definite about something.
The expression may have been used prior to 1951, but made a notable debut in print when Herman Wouk’s Cain Mutiny was published and became a Pulitzer Prize winner.
Lt. Commander Queeg said:
I am damn well responsible for anything that happens on this ship.
From here on in, I don’t expect to make a single mistake.
I won’t tolerate anybody making and mistakes for me, and I “kid you not”. And, well, I think you get the idea without my drawing you a picture.