Well, you're alive aren't you? So didn't screw up TOO badly.
Seriously, if it was simply you tensioning it with the Z-rig, doubtful you overloaded it.
General rule of thumb is you don't want to exceed about 120 degree angle.
Best case, would be 0 degree (this would look like a rope hanging down and up again, each leg supports 1/2 the load.)
The "flatter" it is (i.e. closer to 180 degree) the forces go up dramatically. Useful thing I found to do when I set up a highline was to take a few pictures and then later took a protractor to them and measured. After that exercise, realized I was flatter than I would have liked.
I want to do a few more highlines and take photos so I get better at it. Just have to find the time and equipment.
Hmm, just dawned on me too, that you may want to look at how high you rig on the trees. Even if the rope is not under too much tension, you can pull trees down this way unless properly backtied.