Fengyi Li: Wide Area Measurements of Voice Over IP Quality

Student: Fengyi Li
Thesis: Wide Area Measurements of Voice Over IP Quality
University: KTH
Year: 2003
Now at: Axway Nordic, Stockholm

Links

About the thesis

Programme: MSc, KTH, 2003
Supervisor: Ian Marsh (SICS)
Published as: I. Marsh and F. Li, Wide Area Measurements of Voice Over IP Quality, SICS Technical Report T2003:08, May 2003

The thesis is available on DiVA as this SICS technical report, written with Ian Marsh; the summary below follows it.

Users of real-time voice are sensitive to variable quality: if a call sounds bad, they hang up and try later. As the Internet carried more voice traffic, how good was it, between real end points around the world? The answer matters to many groups: users choosing where to add robustness such as forward error correction, operators deciding where to upgrade or add QoS, regulators checking agreed quality, codec designers building for bursty loss, and researchers.

Method

  • Sites: nine academic hosts in a full mesh: Massachusetts, Michigan and California in the US; Belgium, Finland, Sweden and Germany in Europe; plus Turkey and Argentina.
  • Calls: every hour, each host sent a pre-recorded call to every other, as an RTP stream.
  • Metrics: delay, jitter and loss as defined for RTP (RFC 1889), plus hop counts from traceroute and time-zone differences.
  • Duration: 15 weeks, over 18,000 sessions (18,054 calls in the summary table).
The nine sites in 2002 and the six used in 1998
Measurement sites in 2002 (circles) and in the earlier 1998 study (squares); three were common to both.

Results

Overall quality was good. Across all calls, mean delay was 136 ms, just under the ITU's G.114 recommendation for end-to-end delay but leaving little room for end-system encoding and playout; mean jitter was 4.1 ms (a small buffer absorbs it) and loss 1.8%, acceptable with PCM coding.

Two clear groups:

  • US and Europe (and between them): average delay 105 ms, jitter 3.76 ms, loss 1.16%. Very good.
  • Outside: the Turkish site had large delays, because the Turkish research network reached GÉANT over a satellite link to Belgium, though its jitter (5.7 ms) and loss (4%) were modest. The Argentinian site suffered asymmetry: sending to it was much worse than receiving from it, with a third more delay, twice the jitter and nearly four times the loss.

Losses were mostly single. 78% of all losses were isolated packets, which loss concealment handles well.

Summary of 18,000 VoIP sessions between the nine sites
Delay (D), jitter (J), loss (L), hops (H) and time difference (T) for every sender–receiver pair, with means per site and overall.

Compared with 1998, quality had improved marginally, which the authors attribute to more capacity and better-managed networks.

Conclusions

VoIP quality was acceptable for all but one of the nine sites, and in most cases exceeded the requirements of speech-quality recommendations. Two cautions: all sites were well-provisioned academic networks, and the test flows were small and hourly. Quality depends on the network infrastructure, not just geographic distance, as the contrast between Turkey and Argentina shows.

The paper also notes how hard such measurements are in practice: different access methods, firewalls, NATs and missing permissions all complicate collecting and validating data. The data set was made public through the EU COST 263 action.

Published work from the campaign: A VoIP Measurement Infrastructure (NTS 16, 2002), Wide Area Measurements of Voice Over IP Quality Revisited (RVK 2002) and Wide Area Measurements of VoIP Quality (QoFIS 2003).

Thesis

Full text (DiVA)

Original entry

Fengyi Li, Wide Area Measurements of Voice Over IP Quality, 2003, KTH, now at Axway Nordic, Stockholm, paper, thesis.

Project

Related publications

BibTeX

@mastersthesis{li2003wide,
  title     = {Wide Area Measurements of Voice Over IP Quality},
  author    = {Fengyi Li},
  school    = {KTH},
  year      = {2003}
}

Related publication (no thesis record in DiVA):

@techreport{Marsh1041589,
  author      = {Marsh, Ian and Li, Fengyi and Karlsson, Gunnar},
  title       = {Wide Area Measurements of Voice Over IP Quality},
  institution = {RISE - Research Institutes of Sweden (2017-2019), SICS},
  publisher   = {Swedish Institute of Computer Science},
  series      = {SICS Technical Report},
  number      = {T2003:08},
  ISSN        = {1100-3154},
  pages       = {10},
  year        = {2003},
  url         = {http://www.diva-portal.org/smash/record.jsf?pid=diva2:1041589},
  keywords    = {VoIP, Measurement, loss, delay, jitter},
  edition     = {1}
}

Source: DiVA record diva2:1041589

More in Masters theses: ← Juan Carlos Martín Severiano (2004)  |  Anders Andersson (2000) →

← All supervised theses