Volpe Center Review of CLIA Report

small DOT spiral logo

Memorandum

U.S. Department
of Transportation
Research and
Innovative Technology
Administration
Subject: Review of “PVAG Access Scoping Economic Impact Study”, by Mr. Thomas Gray, on behalf of Cruise Lines International Association (CLIA) Date: February 18, 2010
From: Volpe National Transportation Systems Center
Economic and Industry Analysis Division, RVT-21
Reply to: Catherine Taylor
617-494-2380
To: U.S. Access Board

This memo presents our comments on the “PVAG Access Scoping Economic Impact Study” prepared by Mr. Thomas Gray, on behalf of Cruise Lines International Association (CLIA).  The data used for the report are not available for review and all points are based on the information presented in the CLIA report.

Concerns with this study are categorized under four broad groupings.

  1. Concerns regarding sample selection
  2. Concerns regarding theoretical underpinnings
  3. Concerns regarding forecasting techniques
  4. Concerns regarding economic analysis

1.  Concerns regarding sample selection.

The CLIA study uses two data sources for its analysis.  The first is a sample of 45 ships and their voyages from 2005.  The dataset contains the size of the ship, the number of voyages taken in 2005, the number of passengers, and the use of accessible and non-accessible cabins by passengers using wheelchairs.  The data cover 2,209 voyages.  The study shares no details on how the 45 ships were selected for inclusion in the sample.  Any sort of reliable statistical analysis should be based on a randomized sample.  If the sample selection is biased in any way (and the bias is not accounted for), the results would not be applicable to a larger population.  It seems certain that the sample is not random and does not represent the entire fleet.  The last three lines of Table 4 present total number of cabins available and total accessible cabins for the ships in the sample, the ships outside the sample, and the U.S. cruise fleet as a whole.  Table 4 shows that 2.0 percent of cabins in the sample are accessible (786 of 39,498).  In the fleet as a whole, just 0.8 percent of cabins are accessible (1,813 of 225,364).  Thus, it is obvious that sample of ships used in the study are not representative of the whole fleet.[1]

The second database is longitudinal, containing data on the number of passengers using wheelchairs during the voyages of 15 to 20 ships during the period 2000 through 2006.  Again, the study provides no details on how the ships were selected for inclusion in the study.  In the absence of any information on how the sample was selected and given the obvious bias in the other sample, similar bias may be present in the longitudinal database as well.

Further, when presenting data from these sources, the study opts to present just some of the data points.  Table 2 shows the data for just half the ships in the 2005 database.  Table 3 shows the data for just 5 of the ships in the 2005 database, and of the five shown, just 3 are also shown in Table 2.  It is not clear why the study does not present the data in its entirety.  And some of the study’s inferences appear to be based on just half of the sample.

These problems alone are enough reason to reject the entire study.  However, for completeness sake, the other issues relating to this study will be addressed.

2.  Concerns regarding theoretical underpinnings.

First, the analysis often concerns passengers when the unit of observation should be cabins.  For instance, the discussion surrounding Charts 4 and 5 notes that given the study’s projections,[2] “…usage at the three percent level would not be reached for another 28 years.”  The usage refers to percent of passengers using wheelchairs; however, using a reasonable assumption that each wheelchair-using passenger is sharing a cabin with a non-wheelchair-using partner, then we should double the projected percent of wheelchair-using passengers to arrive at a forecast of cabins containing a wheelchair-using passenger.[3]  We would infer that cabins containing a wheelchair-using passenger would be 6 percent in 28 years.  On Chart 4, it appears that in 17 years wheelchair usage by passengers will reach 1.5 percent, implying that cabins containing wheelchair-using passengers will reach 3 percent in 17 years.  The 17 year time horizon for hitting up against the access board’s recommendation is well within the 30 year average useful life of a cruise ship.[4]

Second, the study focuses on average number of wheelchair-using passengers when the focus should be on the prevalence of voyages with a high number of wheelchair-using passengers.  The study uses the average number of wheelchair users along with an assumption that the number of wheelchair users is distributed normally to back into an estimate of the prevalence of voyages with high number of wheelchair-using passengers.  This methodology is troubling because, a.) the study could have looked at the data directly to see how often 3 percent of cabins housed a wheelchair-using passenger and b.) the assumption of normality is clearly not defensible.  The normal distribution is used when the observed data points can take on any value from -∞ to +∞ and the data points are arranged symmetrically on either side of the mean value.  This is clearly not the case with counts of cabins containing wheelchair users.  This range is bounded by zero on the left and the number of available cabins on the right.  The study should have accounted for the truncated nature of the data in the analysis.  In addition, several statistical procedures can be used to test if the data is in fact distributed normally and the study did not use them.[5]

The study asserts that Chart 6 supports the assumption that the presence of wheelchair using passengers is distributed normally.  However, the information found in Chart 6 is not sufficient to test for normality.  As stated above, formal statistical tests should be used.  Further, the study misrepresents the contents of Chart 6 stating, “fewer than one percent of the observations lie outside the confidence interval defined by the mean plus or minus 2 standard deviations.”  However, Chart 6 actually shows that 3.76 percent (83 out of 2239 voyages) had a number of wheelchair users greater than 2 standard deviations from the mean.  As the study notes, if the number of wheelchair users is normally distributed, 5 percent of records should be outside 2 standard deviations of the sample mean.  Looking at just the upper tail, 2.5% of records should be greater than 2 standard deviations from the mean.  But in the data presented, 3.76% of voyages had a number of wheelchair users greater than 2 standard deviations from the mean.  This suggests that the study’s (untested) assumption of a normal distribution is underestimating the probability of voyages with high number of wheelchair users.

When the study focuses on just wheelchair-using passengers in accessible cabins, it faces another data issue that it does not address.  For some voyages, the number of wheelchair-using passengers in accessible cabins may be constrained by the number of accessible cabins on the ship.  The study gives no information on how often all the available accessible cabins were used by wheelchair-using passengers.  Such information is highly relevant to the issue at hand and its omission is quite surprising.  This constraint could certainly effect the calculation of the average.  For example, consider a case where a ship makes 4 voyages and has 10 accessible cabins.  In 3 of the voyages, 15 passengers needed accessible cabins; in the fourth voyage no passengers needed an accessible cabin.  The data would show that on average only 7.5 of the 10 accessible cabins were used by passengers using wheelchairs (10+10+10/4) and an analyst might assume that no further accessible cabins were needed.  Clearly such an assumption would be incorrect.  Based on the hypothetical scenario, it would seem that 15 accessible cabins would be needed but the data analysis method used by the study would not show that need.

The study discusses the phenomena of wheelchair-using passengers not using accessible cabins.  As illustrated above, this may result when there are not enough accessible cabins available.  The study does not address this possible explanation.

In footnotes 15 and 20 the study states that in any given voyage there may not be enough cabins for all wheelchair-using passengers who desire them but that even passengers who do not use wheelchairs may find that there is not a cabin available for them on their preferred voyage.  These statements are close to heart of the matter.  The question is whether wheelchair-using passengers who desire or need an accessible cabin are denied accessible cabins at a rate that is higher than the general population.  The study’s analysis provides no insight into this basic question.

3.  Concerns regarding forecasting techniques

The study does not provide details of the methodology for forecasting the percent of passengers using wheelchairs but it appears that the study has used a straight linear projection based on the 7 years of available data.  This method is not very sophisticated but it appears to be overestimating the growth in wheelchair-using passengers so in effect the method might be conservative from the industry’s viewpoint.

The study only uses data from the first half of each year because at the time of the analysis data for the second half of 2006 was not available.  However, footnote 16 states that the second half of the year sees more wheelchair users but that the increase in other passengers offsets the relative increase in wheelchair users to some degree.  The study does not claim that the offset is complete, therefore we can infer that the wheelchair usage rates shown in Charts 1 and 3 are underestimating the actual use of wheelchairs.  Thus, the study’s conclusions regarding when wheelchair usage of passengers will hit certain levels are incorrect.

The greater issue lies in the study’s subsequent forecast of percent of wheelchair-using passengers using accessible rooms.  There is no longitudinal data on which type of cabins the wheelchair-using passengers are using so the study assumes that the ratio of wheelchair-using passengers using accessible cabins to non-accessible cabins remains at the 2005 level observed in the 2005 data.  In fact there is no information on how this measure is changing over time so the forecasts are based on nothing but an assumption with no statistical validity.  The lower lines on Charts 3 and 5 should be entirely discounted.

4.  Concerns regarding economic analysis

The report states for every two accessible cabins added to a ship, one regular sized cabin would be lost.  However, this may not be the case where larger suites are converted to be accessible.  The report also states that a cabin is worth $400 in revenue per day.  First, no details are given on how this figure was calculated.  Second, the metric of interest should be the incremental profit of a cabin.  In cases where cruise tickets include airfare or special excursions, the cost to the cruise line of providing those services should be netted out.  Other incremental costs such as food, staff, supplies, electricity, laundry, and water that would be reduced with fewer passengers should also be netted out of the incremental revenue figure.

Further, economic theory suggests that if supply is decreased, prices should rise. Therefore, the rates on the remaining cabins might go up.  In addition, the larger accessible cabins might hold more people who will still need to purchase tickets and may make additional onboard expenditures (gambling, alcohol, photography, etc), providing higher than average revenue per cabin.  For all those reasons, the assumption of $400 loss per cabin per day is not justified.

The study uses many techniques to present figures of lost revenue that appear very large, while neglecting to report the actual figures that would be relevant for this analysis.  First, the proposed regulations do not call for retro-fitting existing ships.  Second, most if not all ships already have some accessible cabins, so including a “loss” for cabins that already exist is disingenuous.  In addition, the study appears to be asserting that any accessible cabin results in a loss of revenue for the cruise line.  Given that cruise ships were not required to have any cabins accessible, and as cruise lines have opted to provide at least some accessible cabins, it appears that the cruise lines do in fact find that having at least some accessible cabins is a profitable activity.

The issue has narrowed to a question of 2 percent or 3 percent of cabins being accessible.  For a ship of 1000 cabins, this difference boils down to just 5 “lost” cabins.[6]  Using the study’s own figures, the difference between requiring 2 and 3 percent would be approximately $566,000 per ship per year.  To put this figure in perspective, Carnival Cruise lines had revenues per ship of $166.432 million in 2008.[7]  For Carnival, the revenue “loss” would be just 0.34 percent per ship per year. 

The study’s analysis of how the revenue losses due to additional accessible cabins will change over time includes no analysis on changes in fares, changes in capacity or the appropriate discount rate to use for the industry.  Further, the study assumes a smooth linear growth in capacity which is not an accurate representation of how capacity would enter the market.  Capacity additions are “lumpy”; a new ship entering the market would create a sizable jump in the available cabins.  Likewise, a ship retirement would cause a discrete step down in capacity. Information on new ships under construction is easily available and could have been included in the analysis.

The study states that the assumptions of 3 percent increase in cabins per year and 3 percent increase in revenue per cabin per year “appear reasonable given past experience.”  The study does not include any data or information that would support this assertion of “reasonableness.”  The calculations are based solely on unsupported assumptions.  In response to the non sequitur question posed in footnote 32: yes, “the whole thing is not statistically valid.”


[1] The 0.8 percent from the fleet population does not fall within the 95 percent confidence interval around the sample estimate of 2.0 percent.

[2] Concerns regarding the study’s forecasting methodology will be discussed below.

[3] To illustrate, suppose there are 100 passengers on a ship with 50 cabins.  If ten of the passengers use a wheelchair and have partners who do not use a wheelchair, then 10 percent of passengers would be wheelchair-using (10 out of 100), but 20 percent of cabins would contain a wheelchair-using passenger (10 out of 50).

[4] Carnival Cruise Annual Report, 2008.

[5] A Q-Q plot, Kolmogorov-Smirnov test, or Shapiro-Wilk test

[6] As discussed above, when larger suites are converted to be accessible there may in fact be no loss in cabins.

[7] Carnival Cruise Annual Report, 2008.