Attach alligator clips to your meter prongs - set your meter to DC voltage and get a reading.
The harness is a very good idea. However, in order for the reading to be accurate, you must be using a meter with a Battery Test function, or you must test the battery under load.
A standard DC voltage meter will only provide a "surface charge" reading.