UNC Network Data Analysis Study Group 
Summary Page for LRD Project

The following statistical techniques are used to study the long-range dependence of network traffic:

1. Aggregated variance
2. Whittle method (this is an excerpt from Prof Richard Smith's talk on October 29th 2002)
3. Wavelet analysis

The data is available from the following web pages:

1. Real data: data taken at the UNC campus link.
2. Synthetic data: data generated by web traffic generators simulating users browsing the web.

For easy access, the data sets are also linked in the left hand column of the tables of results given here.

We apply the aforementioned techniques to both real network data taken at the UNC campus link and synthetic data generated in our lab. The table below contains the estimation of the Hurst parameter from those techniques. Each entry in the table also contains a link to the detailed plots for the estimates.

The plots of the Whittle method are by courtesy of Prof. Richard Smith. The Hurst parameter from the Whittle method was obtained by a course estimation of the confidence intervals on d (where H = d + 1/2) from visual inspection of the plots. Please refer to the plots of the d parameter by following the links in the Whittle method column.

The Hurst parameters from the wavelet analysis shown below are obtained from data sampled at 1-ms intervals. Results from higher aggregation levels are also available for synthetic data and real data. The numbers in parentheses represent the confidence interval for the wavelet analysis.

While most of the data files have only a single column, some have two columns of data, which give byte and packet counts. In all analyses here we consider only the packet counts (second column).

The following results are compared using scatterplots in the web page Comparative Plots of Hurst Parameter Estimation Methods.
 

Automatic
 

Data	Aggregated variance	Whittle method (Richard)	Whittle method (Michele)	Wavelet method	SiZer plots
18 clients, 10 servers, 20 Mbps	H = 0.82	H = 0.899 (0.848 - 0.950)	H =0.8329 (0.8169 - 0.8490)	H = 0.86 (0.82 - 0.89)	SiZer plot
18 clients, 10 servers, 50 Mbps	H = 0.83	H = 0.880 (0.831 - 0.928)	H =0.8481 (0.8166 - 0.8795)	H = 0.87 (0.83 - 0.90)	SiZer plot
18 clients, 10 servers, 80 Mbps	H = 0.84	H = 0.845 (0.801 - 0.890)	H =0.8294 (0.8001 - 0.8587)	H = 0.84 (0.81 - 0.87)	SiZer plot
18 clients, 10 servers, 110 Mbps	H = 0.85	H = 0.900 (0.854 - 0.946)	H =0.8570 (0.8193 - 0.8947)	H = 0.90 (0.85 - 0.96)	SiZer plot
18 clients, 10 servers, 140 Mbps	H = 0.83	H = 0.852 (0.806 - 0.898)	H =0.8342 (0.8163 - 0.8522)	H = 0.82 (0.80 - 0.85)	SiZer plot
Real data: Mon 21:30 April 08 2002	H = 0.86	H = 0.969 (0.922 - 1.015)	H = 0.99 (0.96 - 1.03)	H = 0.99 (0.95 - 1.03)	SiZer plot
Real data: Tue 03:00 April 09 2002	H = 0.95	H = 1.281 (1.229 - 1.332)	H = 1.26 (1.22 - 1.29)	H = 1.14 (1.10 - 1.18)	SiZer plot
Real data: Tue 05:00 April 09 2002	H = 0.86	H = 1.070 (1.020 - 1.119)	H = 1.05 (1.02 - 1.08)	H = 1.02 (0.96 - 1.08)	SiZer plot
Real data: Tue 08:00 April 09 2002	H = 0.98	H = 1.258 (1.214 - 1.301)	H = 1.18 (1.15 - 1.21)	H = 0.95 (0.91 - 0.99)	SiZer plot
Real data: Tue 13:00 April 09 2002	H = 0.90	H = 1.009 (0.970 - 1.047)	H = 0.97 (0.95 - 1.00)	H = 0.91 (0.88 - 0.93)	SiZer plot
Real data: Tue 19:30 April 09 2002	H = 0.86	H = 0.983 (0.938 - 1.028)	H = 0.96 (0.93 - 0.99)	H = 0.91 (0.89 - 0.94)	SiZer plot
Real data: Wed 13:00 April 10 2002	H = 0.87	H = 0.978 (0.936 - 1.020)	H = 0.98 (0.95 - 1.01)	H = 0.95 (0.91 - 0.99)	SiZer plot
Real data: Wed 21:30 April 10 2002	H = 0.85	H = 0.981 (0.935 - 1.026)	H = 0.97 (0.94 - 1.01)	H = 0.98 (0.94 - 1.02)	SiZer plot
Real data: Thu 03:00 April 11 2002	H = 0.92	H = 1.093 (1.052 - 1.134)	H =1.03 (1.01 - 1.06)	H = 0.96 (0.92 - 1.00)	SiZer plot
Real data: Thu 10:00 April 11 2002	H = 0.94	H = 1.085 (1.044 - 1.125)	H = 1.02 (1.00 - 1.05)	H = 0.96 (0.92 - 1.00)	SiZer plot
Real data: Thu 13:00 April 11 2002	H = 0.87	H = 0.828 (0.781 - 0.875)	H = 1.06 (1.02 - 1.11)	H = 0.79 (0.50 - 1.09)	SiZer plot
Real data: Thu 15:00 April 11 2002	H = 0.86	H = 0.948 (0.905 - 0.992)	H = 0.95 (0.92 - 0.99)	H = 0.91 (0.88 - 0.94)	SiZer plot
Real data: Thu 19:30 April 11 2002	H = 0.86	H = 0.970 (0.923 - 1.017)	H = 0.98 (0.95 - 1.01)	H = 0.94 (0.92 - 0.97)	SiZer plot
Real data: Fri 03:00 April 12 2002	H = 0.96	H = 1.183 (1.142 - 1.224)	H = 1.12 (1.09 - 1.14)	H = 1.00 (0.94 - 1.06)	SiZer plot
Real data: Fri 21:30 April 12 2002	H = 0.85	H = 0.987 (0.942 - 1.032)	H = 0.94 (0.91 - 0.97)	H = 0.95 (0.91 - 0.99)	SiZer plot
Real data: Sat 10:00 April 13 2002	H = 0.90	H = 1.000 (0.960 - 1.040)	H = 1.01 (0.98 - 1.03)	H = 0.94 (0.77 - 1.10)	SiZer plot
Real data: Sat 13:00 April 13 2002	H = 0.94	H = 1.222 (1.183 - 1.261)	H = 1.11 (1.08 - 1.13)	H = 1.48 (1.32 - 1.64)	SiZer plot
Real data: Sat 15:00 April 13 2002	H = 0.89	H = 1.029 (0.986 - 1.071)	H = 1.00 (0.97 - 1.03)	H = 1.00 (0.94 - 1.06)	SiZer plot
Real data: Sat 19:30 April 13 2002	H = 0.83	H = 0.906 (0.862 - 0.951)	H = 0.92 (0.89 - 0.95)	H = 0.89 (0.87 - 0.92)	SiZer plot
Real data: Sat 21:30 April 13 2002	H = 0.86	H = 1.011 (0.964 - 1.059)	H = 0.99 (0.95 - 1.02)	H = 0.94 (0.91 - 0.96)	SiZer plot

"Tuned"

Data	Aggregated variance	Whittle method	Whittle method (Michele)	Wavelet method	SiZer plots
18 clients, 10 servers, 20 Mbps	H = 0.82	H = (0.8 - 0.88)	H =0.8542 (0.8163 - 0.8922)	H = 0.84 (0.81 - 0.86)	SiZer plot
18 clients, 10 servers, 50 Mbps	H = 0.82	H = (0.8 - 0.88)	H =0.8728 (0.8341 - 0.9114)	H = 0.90 (0.85 - 0.96)	SiZer plot
18 clients, 10 servers, 80 Mbps	H = 0.83	H = (0.8 - 0.88)	H =0.8488 (0.8039 - 0.8937)	H = 0.85 (0.82 - 0.88)	SiZer plot
18 clients, 10 servers, 110 Mbps	H = 0.84	H = (0.8 - 0.86)	H =0.9014 (0.8554 - 0.9474)	H = 0.88 (0.82 - 0.94)	SiZer plot
18 clients, 10 servers, 140 Mbps	H = 0.82	H = (0.8 - 0.85)	H =0.8596 (0.8138 - 0.9053)	H = 0.82 (0.80 - 0.85)	SiZer plot
Real data: Mon 21:30 April 08 2002	H = 0.91	H = (0.88 - 0.93)	H = 0.99 (0.96 - 1.03)	H = 1.00 (0.96 - 1.04)	SiZer plot
Real data: Tue 03:00 April 09 2002	H = 0.98	H = (0.87 - 0.92)	H = 1.26 (1.22 - 1.29)	H = 1.06 (1.01 - 1.10)	SiZer plot
Real data: Tue 05:00 April 09 2002	H = 0.93	H = (0.86 - 0.92)	H = 1.06 (1.03 - 1.09)	H = 1.07 (1.00 - 1.14)	SiZer plot
Real data: Tue 08:00 April 09 2002	H = 0.99	H = (1.0 - 1.06)	H = 1.25 (1.21 - 1.29)	H = 0.93 (0.89 - 0.98)	SiZer plot
Real data: Tue 13:00 April 09 2002	H = 0.96	H = (0.88 - 0.92)	H = 1.03 (0.99 - 1.08)	H = 0.91 (0.88 - 0.93)	SiZer plot
Real data: Tue 19:30 April 09 2002	H = 0.91	H = (0.9 - 0.94)	H = 0.98 (0.94 - 1.03)	H = 0.95 (0.91 - 0.99)	SiZer plot
Real data: Wed 13:00 April 10 2002	H = 0.93	H = (0.87 - 0.92)	H = 0.98 (0.95 - 1.02)	H = 0.97 (0.93 - 1.01)	SiZer plot
Real data: Wed 21:30 April 10 2002	H = 0.91	H = (0.88 - 0.93)	H = 0.98 (0.93 - 1.02)	H = 0.98 (0.94 - 1.02)	SiZer plot
Real data: Thu 03:00 April 11 2002	H = 0.98	H =	H =1.03 (1.01 - 1.06)	H = 0.97 (0.93 - 1.02)	SiZer plot
Real data: Thu 10:00 April 11 2002	H = 0.98	H =	H = 1.04 (1.01 - 1.07)	H = 1.00 (0.93 - 1.07)	SiZer plot
Real data: Thu 13:00 April 11 2002	H = 0.93	H =	H = 1.10 (1.08 - 1.13)	H = 0.82 (0.30 - 1.34)	SiZer plot
Real data: Thu 15:00 April 11 2002	H = 0.90	H =	H = 0.95 (0.90 - 0.99)	H = 0.91 (0.88 - 0.94)	SiZer plot
Real data: Thu 19:30 April 11 2002	H = 0.91	H =	H = 0.97 (0.92 - 1.02)	H = 0.96 (0.93 - 0.98)	SiZer plot
Real data: Fri 03:00 April 12 2002	H = 0.99	H =	H = 1.18 (1.14 - 1.22)	H = 1.04 (0.98 - 1.11)	SiZer plot
Real data: Fri 21:30 April 12 2002	H = 0.91	H =	H = 0.94 (0.91 - 0.97)	H = 0.94 (0.90 - 0.98)	SiZer plot
Real data: Sat 10:00 April 13 2002	H = 0.96	H =	H = 1.02 (0.98 - 1.06)	H = 0.91 (0.86 - 0.95)	SiZer plot
Real data: Sat 13:00 April 13 2002	H = 0.98	H =	H = ??? (??? - ???)	H = 1.49 (1.11 - 1.87)	SiZer plot
Real data: Sat 15:00 April 13 2002	H = 0.95	H =	H = 1.03 (0.99 - 1.07)	H = 0.97 (0.93 - 1.01)	SiZer plot
Real data: Sat 19:30 April 13 2002	H = 0.87	H =	H = 0.91 (0.86 - 0.96)	H = 0.90 (0.88 - 0.93)	SiZer plot
Real data: Sat 21:30 April 13 2002	H = 0.91	H =	H = 0.99 (0.95 - 1.02)	H = 0.98 (0.91 - 1.06)	SiZer plot

Detrended data

Data	Aggregated variance	Whittle method (Richard)	Whittle method (Michele)	Wavelet method	SiZer plots
18 clients, 10 servers, 20 Mbps	H = 0.82	H = 0.899 (0.848 - 0.950)	H =0.8332 (0.8175 - 0.8489	H = 0.86 (0.82 - 0.89)	SiZer plot
18 clients, 10 servers, 50 Mbps	H = 0.83	H = 0.879 (0.830 - 0.928)	H =0.8617 (0.8312 - 0.8922)	H = 0.87 (0.83 - 0.90)	SiZer plot
18 clients, 10 servers, 80 Mbps	H = 0.84	H = 0.845 (0.800 - 0.890)	H =0.8500 (0.8316 - 0.8684)	H = 0.84 (0.81 - 0.87)	SiZer plot
18 clients, 10 servers, 110 Mbps	H = 0.85	H = 0.901 (0.854 - 0.947)	H =0.9078 (0.8759 - 0.9397)	H = 0.90 (0.85 - 0.96)	SiZer plot
18 clients, 10 servers, 140 Mbps	H = 0.83	H = 0.852 (0.806 - 0.897)	H =0.8331 (0.8149 - 0.8513)	H = 0.82 (0.80 - 0.85)	SiZer plot
Real data: Mon 21:30 April 08 2002	H = 0.86	H = 0.969 (0.922 - 1.016)	H = 1.00 (0.96 - 1.03)	H = 0.99 (0.95 - 1.03)	SiZer plot
Real data: Tue 03:00 April 09 2002	H = 0.95	H = 1.280 (1.228 - 1.332)	H = 1.26 (1.22 - 1.29)	H = 1.14 (1.10 - 1.18)	SiZer plot
Real data: Tue 05:00 April 09 2002	H = 0.86	H = 1.069 (1.019 - 1.119)	H = 1.05 (1.02 - 1.08)	H = 1.02 (0.96 - 1.08)	SiZer plot
Real data: Tue 08:00 April 09 2002	H = 0.91	H = 1.096 (1.051 - 1.141)	H = 1.22 (1.19 - 1.24)	H = 0.95 (0.91 - 0.99)	SiZer plot
Real data: Tue 13:00 April 09 2002	H = 0.86	H = 0.960 (0.916 - 1.003)	H = 0.99 (0.97 - 1.02)	H = 0.91 (0.88 - 0.93)	SiZer plot
Real data: Tue 19:30 April 09 2002	H = 0.86	H = 0.983 (0.939 - 1.028)	H = 0.96 (0.93 - 0.99)	H = 0.91 (0.89 - 0.94)	SiZer plot
Real data: Wed 13:00 April 10 2002	H = 0.84	H = 0.950 (0.906 - 0.995)	H = 0.99 (0.96 - 1.02)	H = 0.95 (0.91 - 0.99)	SiZer plot
Real data: Wed 21:30 April 10 2002	H = 0.85	H = 0.977 (0.931 - 1.023)	H = 0.97 (0.94 - 1.01)	H = 0.98 (0.94 - 1.02)	SiZer plot
Real data: Thu 03:00 April 11 2002	H = 0.84	H = 0.982 (0.936 - 1.028)	H =1.07 (1.04  -1.10)	H = 0.96 (0.92 - 1.00)	SiZer plot
Real data: Thu 10:00 April 11 2002	H = 0.87	H = 1.006 (0.962 - 1.051)	H = 1.05 (1.02 - 1.08)	H = 0.96 (0.92 - 1.00)	SiZer plot
Real data: Thu 13:00 April 11 2002	H = 0.86	H = 0.802 (0.752 - 0.852)	H = 1.07 (1.02 - 1.11)	H = 0.79 (0.50 - 1.09)	SiZer plot
Real data: Thu 15:00 April 11 2002	H = 0.85	H = 0.939 (0.894 - 0.983)	H = 0.96 (0.93 - 0.99)	H = 0.91 (0.88 - 0.94)	SiZer plot
Real data: Thu 19:30 April 11 2002	H = 0.86	H = 0.966 (0.919 - 1.013)	H = 0.98 (0.95 - 1.01)	H = 0.94 (0.92 - 0.97)	SiZer plot
Real data: Fri 03:00 April 12 2002	H = 0.84	H = 1.031 (0.983 - 1.079)	H = 1.16 (1.13 - 1.18)	H = 1.00 (0.94 - 1.06)	SiZer plot
Real data: Fri 21:30 April 12 2002	H = 0.85	H = 0.987 (0.942 - 1.033)	H = 0.94 (0.91 - 0.97)	H = 0.95 (0.91 - 0.99)	SiZer plot
Real data: Sat 10:00 April 13 2002	H = 0.84	H = 0.919 (0.875 - 0.963)	H = 1.03 (1.00 - 1.06)	H = 0.94 (0.77 - 1.10)	SiZer plot
Real data: Sat 13:00 April 13 2002	H = 0.94	H = 1.222 (1.183 - 1.261)	H = 1.11 (1.08 - 1.13)	H = 1.48 (1.32 - 1.64)	SiZer plot
Real data: Sat 15:00 April 13 2002	H = 0.89	H = 1.029 (0.986 - 1.072)	H = 1.01 (0.98 - 1.04)	H = 1.00 (0.94 - 1.06)	SiZer plot
Real data: Sat 19:30 April 13 2002	H = 0.83	H = 0.906 (0.861 - 0.951)	H = 0.92 (0.89 - 0.96)	H = 0.89 (0.87 - 0.92)	SiZer plot
Real data: Sat 21:30 April 13 2002	H = 0.86	H = 1.011 (0.964 - 1.059)	H = 0.99 (0.95 - 1.02)	H = 0.94 (0.91 - 0.96)	SiZer plot

Simulated data (automatic)

Data	Aggregated variance	Whittle method (Michele)	Wavelet method	SiZer plots
Fractional Gaussian Noise (Spectral)	H = 0.87	H =0.8998 (0.8944 - 0.9051)	H = 0.90 (0.90 - 0.90)	SiZer plot
Fractional Brownian Motion (Spectral)	H = 1.00	H =1.5013 (1.4958 - 1.5068)	H = 1.90 (1.90 - 1.90)	SiZer plot
Fractional Gaussian Noise (Wavelet)	H = 0.86	H =0.9001 (0.8988 - 0.9014)	H = 0.90 (0.90 - 0.90)	SiZer plot
Fractional Gaussian Noise (Fourier)	H = 0.87	H =0.9010 (0.8991 - 0.9030)	H = 0.90 (0.90 - 0.90)	SiZer plot

Simulated data ("tuned")

Data	Aggregated variance	Whittle method (Michele)	Wavelet method	SiZer plots
Fractional Gaussian Noise (Spectral)	H = 0.89	H =0.8966 (0.8721 - 0.9210)	H = 0.90 (0.90 - 0.90)	SiZer plot
Fractional Brownian Motion (SPectral)	H = 1.0	H =1.5279 (1.5029 - 1.5529)	H = 1.90 (1.90 - 1.90)	SiZer plot
Fractional Gaussian Noise (Wavelet)	H = 0.88	H =0.8712 (0.8383 - 0.9040)	H = 0.90 (0.90 - 0.90)	SiZer plot
Fractional Gaussian Noise (Fourier)	H = 0.88	H =0.8990 (0.8760 - 0.9220)	H = 0.90 (0.90 - 0.90)	SiZer plot

 

Dependent SiZer Web page

Wavelet coefficients + SiZer