Lopes, Leite, Carmo Correia, and Durán-Santomil: Market crises and benchmark-adjusted fund alphas in a small market context

1. INTRODUCTION

The growth of the mutual fund industry has highlighted the importance of correctly evaluating the performance of these financial products. However, the predominance of neutral or below-market performance, which contrasts with the continuous demand for actively managed funds, has triggered a wide debate on the effectiveness of the measures used.

Following the limitations identified regarding classical performance evaluation measures (; ; ), the literature has produced several developments in terms of evaluation models, namely multifactor specifications that incorporate additional risk factors, such as size and book-to-market (), momentum (), and profitability and investment (). Since they are theoretically more robust than single factor models, multifactor ones allow for a better characterization of managers’ investment styles and, consequently, a more accurate assessment of performance. Yet, most studies based on these models still indicate a predominance of negative or neutral performance from fund managers in several international mutual fund markets (e.g., ; ; ). A potential reason for these findings may be that they interpret fund alphas as a manager’s incremental performance in relation to passive benchmark indices, which should present statistically insignificant alphas. However, if this assumption is violated, mutual fund performance estimates are biased.

argue that, even in the context of the or models, passive benchmark indices may exhibit statistically significant non-zero alphas, along with significant systematic risk factor coefficients. In line with this argument, suggest using benchmark-adjusted alphas to measure equity fund performance. Through an empirical study of 5,738 US funds, between September 1998 and June 2012, the authors report an average 4-factor alpha of -2.11% per year, while the adjusted alpha is only -1.25% per year, on average. This result suggests that benchmark-adjusted alphas may lead to an improvement in fund performance estimates.

also find that the methodology of brings the fund alphas closer to zero. They propose an optimization algorithm that adjusts the four factors in order to achieve an alpha of zero for any benchmark index. For a total sample of 1,383 equity funds from the US market, the adjustment leads to higher (lower) fund alphas when benchmark (S&P500) alphas are negative (positive).

Using a sample of 887 UK funds from the period 1992-2013, show that benchmark-adjusted alphas are significantly higher than traditional ones for both 3-factor () and 4-factor () models, with differences reaching 127 basis points per year. More importantly, contrary to the majority of past bodies of research, which have reported that UK funds significantly underperformed, the adjusted alphas support evidence of significant outperformance. These results were attributed to the negative benchmark alphas (FTSE 100), which were more noticeable during bear- than during bull-market periods, resulting in fund performance being more (less) undervalued in bear (bull) markets.

In a subsequent study, evaluate the impact of the mismatch between fund objectives and the prospectus benchmark for 1,281 US funds, using the S&P500 as the benchmark. Based on the adjusted-alpha methodology, their results show that, contrary to the traditional model where positive (negative) fund performance is related to a positive (negative) benchmark performance, when the methodology is applied, the relationship is reversed, with a prevalence of negative adjusted alphas in the periods when the benchmark exhibits positive performance, thus changing the perception that investors have of the actual performance of mutual funds.

As a consequence, the literature confirms the importance of considering benchmarks of funds to adequately measure their performance. As mentioned by , larger negative benchmark alphas increase the possibility of mutual fund alphas changing from negative and statistically significant when using traditional models to positive when employing benchmark-adjusted specifications.

Although there are already some practical applications of the methodology (e.g., Mateus et al., , ; ; ), these are restricted to large mutual fund markets, like the US and UK ones. Therefore, one of the main contributions of this paper is that, to the best of our knowledge, it is the first to compute and analyze benchmark-adjusted alphas for funds domiciled in a small market, namely Portugal. This is an interesting research topic because, besides avoiding data mining, benchmark indices in small markets include a considerably lower number of stocks and exhibit higher concentration levels. A more restricted investment universe, combined with lower information costs (), may help fund managers to be better stock pickers and, consequently, to outperform their benchmarks. However, most previous bodies of work that analyze the performance of Portuguese-based equity funds, using distinct sample periods and employing several different performance evaluation methodologies, show that they tend to underperform the market (e.g., ; Leite & Cortez, , ; ). A potential justification for these findings may be that all these studies are based on standard, non-benchmark-adjusted, performance evaluation models, which may not recognize a manager’s superior performance if the benchmark itself presents a consistently negative alpha. Therefore, the performance of Portuguese mutual funds should be re-evaluated using benchmark-adjusted alphas.

Furthermore, unlike the studies of Mateus et al. (, ) and , which focus on investing in domestic mutual funds, our work is one of the first to extend this research topic to making investments in international funds (more precisely, at a European level), for which benchmarks are less concentrated and comprise a considerably higher number of stocks. have also evaluated the benchmark-adjusted performance of US funds that invest in European stocks and find the results to be consistent with those obtained for domestic US funds. However, their work is based only on the 4-factor model of and does not incorporate the more recent investment and profitability factors of .

Another contribution that this piece of work makes is that it divides and compares both benchmark and mutual fund performance across different market conditions. Unlike previous studies that assume a certain calendar year as a bull- or bear- market period (e.g., ), market states are identified using a more robust approach, based on the econometric algorithm of , which is also used by and , among others. Therefore, we aim to evaluate whether the impact of using benchmark-adjusted alphas is higher during bear / crisis or bull / non-crisis phases, for both fund categories in our dataset (domestic and European). To accomplish this, we have used a broad 21-year (2000-2020) evaluation period that covers several different bull- and bear-market phases.

The rest of the paper is structured as follows: section 2 details the methodology; section 3 provides a description of the data; section 4 presents and discusses the results; section 5 presents the conclusions and some suggestions for further research.

2. METHODOLOGY

2.1 Benchmark alphas

We have estimated the alphas of each benchmark using 4-, 5- and 6-factor models, in line with and Fama and French (, ). The 6-factor specification is written as:

R_{b, t} - R_{f, t} = α_{b} + β_{1 b} (R m_{t} - {R f}_{t}) + β_{2 b} {SM B}_{t} + β_{3 b} HM L_{t} + β_{4 b} RM W_{t} + β_{5 b} CM A_{t} + β_{6 b} WM L_{t} + ε_{b, t}

[1]

where 𝑅𝑏_𝑡 − 𝑅𝑓_𝑡 is the excess return (over the risk-free rate) of the benchmark in period t and 𝑅𝑚_𝑡 − 𝑅𝑓_𝑡 is the excess return of a broader market index during the same period. 𝑆𝑀𝐵_𝑡, 𝐻𝑀𝐿_𝑡, 𝑅𝑀𝑊_𝑡 and 𝐶𝑀𝐴_𝑡 are the size, book-to-market, profitability, and investment factors, respectively, and 𝑊𝑀𝐿_𝑡 is the momentum factor. The 5- factor alpha is estimated excluding the momentum factor (𝑊𝑀𝐿_𝑡 ) from equation [1], while the 4-factor alpha is estimated excluding the investment (𝑅𝑀𝑊_𝑡) and profitability (𝐶𝑀𝐴_𝑡) factors from the same equation.

2.2 Unadjusted and benchmark-adjusted fund alphas

Unadjusted and benchmark-adjusted mutual fund alphas are also estimated with 4-, 5- and 6-factor models. The unadjusted (𝛼_𝑝) 6-factor alphas are based on the following equation:

R_{p, t} - R_{f, t} = α_{p} + β_{1 p} (R m_{t} -R f_{t}) + β_{2 p} SM B_{t} + β_{3 p} HM L_{t} + β_{4 p} RM W_{t} + β_{5 p} CM A_{t} + β_{6 p} WM L_{t} + ε_{p, t}

[2]

where 𝑅𝑝_𝑡 − 𝑅𝑓_𝑡 is the excess return of fund p in period t and the remaining variables are as described previously. After this, we estimated the 6-factor benchmark-adjusted alphas ( $α_{p}^{*}$ ) using the following formula:

R_{p, t} - R_{b, t} = α_{p}^{*} + β_{1}^{*} (R_{m, t} - R_{f, t}) + β_{2}^{*} SM B_{t} + β_{3}^{*} HM L_{t} + β_{4}^{*} RM W_{t} + β_{5}^{*} CM A_{t} + β_{6}^{*} WM L_{t} + ε_{p, t}^{*}

[3]

where 𝑅_𝑝,𝑡− 𝑅_𝑏,𝑡 is the excess return of fund p over its benchmark in period t. In equation [3], $α_{p}^{*}$ is the difference between the alpha of the fund (𝛼_𝑝) and the alpha of the benchmark (𝛼_𝑏) and $β_{1}^{*}$ is the difference between the market beta of the fund (𝛽_1𝑝) and the market beta of the benchmark (𝛽 _1𝑏 ). Similar interpretations apply to the following parameters.

2.3 Crisis and non-crisis phases

To investigate whether benchmark and mutual fund performance differ across different market conditions, we began by detecting the crisis / bear-market periods based on the econometric algorithm of . The peaks / troughs of each benchmark index occur when they are the highest / lowest values in an eight-month window of surrounding values. Hence, with Pt representing the value of the index, a trough occurs at time t if 𝑙𝑛(𝑃_𝑡−8,..., 𝑃_𝑡−1) > 𝑙𝑛(𝑃_𝑡) < 𝑙𝑛(𝑃_𝑡+1,..., 𝑃_𝑡+8) and a peak occurs at time t if 𝑙𝑛(𝑃_𝑡−8,..., 𝑃_𝑡−1) < 𝑙𝑛(𝑃_𝑡) > 𝑙𝑛(𝑃_𝑡+1,..., 𝑃_𝑡+8). After this, all falls of at least 20% from peak to trough are classified as crisis / bear-market phases, whereas the remaining periods are considered non-crisis / bull-market phases.

We have used the PSI and the FTSE Eurofirst 100 as they are suitable stock market indices for our study. Over the period 2000-2020, we have identified six crisis periods for the PSI index and four for the FTSE Eurofirst 100 index, which are detailed in Table 1 below.

Table 1Crisis Periods

Start	End	Change in market index	Length of period (months)
Panel A: PSI index
02/2000	09/2002	-65.13%	32
07/2007	02/2009	-55.31%	20
10/2010	05/2012	-44.16%	20
03/2014	12/2014	-36.92%	10
04/2015	06/2016	-26.93%	15
01/2020	03/2020	-22.51%	3
Panel B: FTSE Eurofirst 100 index
Start	End	Change in market index	Length of period (months)
10/2000	03/2003	-56.23%	30
06/2007	02/2009	-55.15%	21
02/2011	09/2011	-21.93%	8
12/2019	03/2020	-25.97%	4

For both indices we can observe crisis periods in the early 2000s, related to the dot-com bubble crash, from the second half of 2007 to the beginning of 2009, associated with the global financial crisis, and during the first quarter of 2020, following the Covid-19 pandemic. The remaining periods (1 for the European index, in 2011, and 3 for the Portuguese index, covering from the last quarter of 2010 to the first half of 2016) are mostly linked to the Euro-area debt crisis, which also impacted the equity sector. As Portugal was one of the most affected countries, requiring an official bailout programme financed by the European Central Bank, the European Commission, and the International Monetary Fund, it may justify why the total length of the crisis period is considerably higher for the Portuguese index (100 months) than for the European index (63 months).

Following , we have obtained alpha and beta estimates in the crisis and non-crisis phases separately by adding two dummy variables to all the previous equations. Based on the 6-factor model, we have obtained the benchmark alphas, the unadjusted fund alphas, and the benchmark-adjusted fund alphas through equations [4], [5] and [6], respectively:

R_{b, t} - R_{f, t} = α_{b,NC} D_{NC} + α_{b,C} D_{C} + \sum_{i = 1}^{6} β_{i b,NC} F_{i, t} D_{NC} + \sum_{i = 1}^{6} β_{i b,C} F_{i, t} D_{c} + ε_{b, t}

[4]

R_{p, t} - R_{f, t} = α_{p,NC} D_{NC} + α_{p,C} D_{C} + \sum_{i = 1}^{6} β_{i p,NC} F_{i, t} D_{NC} + \sum_{i = 1}^{6} β_{i p,C} F_{i, t} D_{C} + ε_{p, t}

[5]

R_{p, t} - R_{b, t} = α_{p,NC}^{*} D_{NC} + α_{p,C}^{*} D_{C} + \sum_{i = 1}^{6} β_{i,NC}^{*} F_{i, t} D_{NC} + \sum_{i = 1}^{6} β_{i,C}^{*} F_{i, t} D_{C} + ε_{p, t}^{*}

[6]

where $F_{i, t}$ represents each of the six risk factors described earlier, $D_{NC}$ ( $D_{C}$ ) equals 1 for the non-crisis (crisis) phases and 0 otherwise. In equation [4], $α_{b,NC}$ and $α_{b,C}$ are the non-crisis and crisis period benchmark alphas, respectively. In equations [5] and [6], $α_{p,NC}$ ( $α_{p,C}$ ) and $α_{p,NC}^{*}$ ( $α_{p,C}^{*}$ ) are the non-adjusted and benchmark-adjusted fund alphas for the non-crisis (crisis) phases, respectively. The remaining parameters are straightforward to interpret.

3. DATA

Our dataset includes all (36) active open-end funds based in Portugal, investing in domestic (10 funds) and European (26 funds) equities, with a minimum of 24 monthly observations from January 2000 to December 2020. Given that non-surviving funds are also included, there is no survivorship bias. By the end of the evaluation period, fund age had reached an average of 15.73 years, fund size was €14.34 million, and the average total expense ratio was 1.85% per year.

End-of-month fund net asset values were collected from CMVM – the Portuguese Securities Market Commission . After analyzing the funds’ prospectuses, and verifying they were all accumulation funds, their returns (R_p) were calculated using the following expression: R_p = (NAV _t – NAV _t-1 ) / NAV _t-1 , where NAV _t (NAV _t-1 ) is the fund’s net asset value at time t (t-1). These returns include all operating expenses, such as management and supervision fees, but exclude subscription and redemption fees.

In Table 2, some descriptive statistics are displayed for the monthly excess returns of two equally weighted fund portfolios, one for each category (domestic and European). The risk-free rate corresponds to the 1-month Euribor rate. On average, both fund categories exhibit negative excess returns for the full sample period. However, European funds not only display lower returns but also lower standard deviations than their domestic equivalents. Despite this, the difference in mean excess returns is highly prominent in the two market states. Although during the crisis (non-crisis) phases the two fund types exhibited negative (positive) mean excess returns, in comparison to European funds, the domestic funds show considerably (slightly) higher returns during the non-crisis (crisis) phases. Besides this, as expected, the highest standard deviations for both portfolios occurred during the market downturns.

Table 2Descriptive Statistics for Fund Portfolios, 2000-2020

		Mean (%)	Median (%)	Max. (%)	Min. (%)	Std. Dev. (%)	Observations
Domestic	Entire period	-0.1919	-0.0395	14.3769	-19.5710	5.4070	251
	Non-crisis	1.8240	1.3673	17.4180	-8.7857	4.3019	151
	Crisis	-2.8558	-2.3981	14.3769	-19.5710	6.0637	100
European	Entire period	-0.3099	0.4155	12.4539	-16.9559	4.5700	251
	Non-crisis	0.9641	1.6822	16.9807	-8.2508	3.7633	188
	Crisis	-3.5327	-2.4385	12.1227	-16.8647	5.6475	63

The fund benchmarks are the PSI total return index for the domestic funds and the FTSE Eurofirst 100 total return index for the European funds. These indices are frequently used to represent Portuguese and European equity market trends. As broader total return market indices we have used the PSI All-Share and the FTSE Europe.

The European (size, book-to-market, investment, profitability, and momentum) factors were obtained from Kenneth French's website, while the domestic factors were collected from Applied Quantitative Research (AQR). All factors were converted to euros using the USD/EUR exchange rate. Factor correlations, ranging from -0.37 and 0.16 for the domestic market and from -0.51 and 0.70 for the European market, allow us to avoid multicollinearity issues.

4. RESULTS

4.1 Benchmark alphas

Since benchmarks should be broad passive indices, they should not generate abnormal returns. However, if a positive (negative) performance is verified, it must be represented by a positive (negative) alpha. Figure 1 presents the alphas obtained for the domestic and European benchmarks, expressed in basis points per year, using 3-year moving averages and the 4-factor model of , for the entire evaluation period.

Figure 1.Evolution of Benchmark Alphas

We have found evidence of consistently negative alphas for both benchmarks for most of the sample period. Comparing the domestic and European benchmarks, we can also see that these negative alphas were not only considerably more frequent, but also of a notably larger magnitude for the PSI than for the FTSE Eurofirst 100.

To explore this topic in greater detail, Table 3 shows performance (monthly and yearly alphas) and risk estimates for the domestic (PSI) and European (FTSE Eurofirst 100) benchmarks in the period 2000-2020. We used three alternative models: the 6-factor model (FF6) of equation [1]; a 5-factor variant (FF5) that omits the momentum factor; a 4-factor version (CH4) that excludes investment and profitability factors. Besides the total sample period, results are also presented for the crisis and non-crisis phases separately, following equation [4].

Table 3 Benchmark Alphas

			Alpha per month (%)	Alpha per year (bps)	Market beta	SMB	HML	RMW	CMA	WML	Adj. R²
PSI	CH4	Total sample	-0.1204*	-143.53	1.0399***	0.0080	-0.0087			-0.0639**	0.9708
			(0.0548)		(0.0000)	(0.6357)	(0.6504)			(0.0166)
		Non-crisis	-0.1071	-127.77	1.0587***	0.0172	-0.0360			-0.0617	0.9711
			(0.1635)		(0.0000)	(0.4026)	(0.1959)			(0.1363)
		Crisis	-0.2495**	-295.33	1.0175***	-0.0144	0.0207			-0.0678***	0.9711
			(0.0276)		(0.0000)	(0.5249)	(0.1706)			(0.0055)
FTSE Eurofirst 100	CH4	Total sample	-0.0930**	-111.03	1.0006***	-0.2157***	0.0568*			-0.0020	0.9849
			(0.0199)		(0.0000)	(0.0000)	(0.0674)			(0.8862)
		Non-crisis	-0.1192***	-142.11	1.0027***	-0.2020***	0.0751**			0.0001	0.9848
			(0.0031)		(0.0000)	(0.0000)	(0.0383)			(0.9946)
		Crisis	-0.0150	-17.99	1.0120***	-0.2187***	0.0324			0.0119	0.9848
			(0.7697)		(0.0000)	(0.0000)	(0.3733)			(0.5740)
	FF5	Total sample	-0.1042**	-124.33	1.0046***	-0.2157***	0.0607	0.0245	0.0050		0.9849
			(0.0122)		(0.0000)	(0.0000)	(0.1903)	(0.4245)	(0.9228)
		Non-crisis	-0.1407***	-167.54	1.0114***	-0.1860***	0.0749	0.0515	0.0519		0.9849
			(0.0007)		(0.0000)	(0.0000)	(0.2038)	(0.3248)	(0.3575)
		Crisis	-0.0284	-34.03	1.0086***	-0.2282***	0.0561	0.0567	-0.0258		0.9849
			(0.6570)		(0.0000)	(0.0000)	(0.3215)	(0.2270)	(0.7213)
	FF6	Total sample	-0.1012**	-120.77	1.0031***	-0.2129***	0.0573	0.0301	0.0099	-0.0070	0.9849
			(0.0140)		(0.0000)	(0.0000)	(0.2031)	(0.3726)	(0.8511)	(0.6614)
		Non-crisis	-0.1389***	-165.41	1.0108***	-0.1854***	0.0718	0.0519	0.0534	-0.0031	0.9848
			(0.0007)		(0.0000)	(0.0000)	(0.2234)	(0.3221)	(0.3492)	(0.8670)
		Crisis	-0.0281	-33.67	1.0099***	-0.2291***	0.0559	0.0529	-0.0273	0.0040	0.9848
			(0.6650)		(0.0000)	(0.0000)	(0.3264)	(0.3337)	(0.7258)	(0.8901)

Note: Standard errors were adjusted for autocorrelation and heteroskedasticity (Newey & West, 1987). P-values are in parentheses

***,**,* indicate the statistically significant coefficients at the 1%, 5%, and 10% levels, respectively

For the total sample period, the results show the existence of negative and statistically significant alphas for both benchmarks, reaching -143.53 basis points per year for the PSI index and between -111.03 and -124.33 basis points per year for the FTSE Eurofirst 100 index, in accordance with and . Therefore, if mutual fund performance is evaluated against these benchmarks, fund alphas will have a downward bias. What is more, in comparison to the broader market index, the European (but not the domestic) benchmark exhibited clear large cap biases, as expected.

The results obtained separately for the market crisis and non-crisis phases show a substantial contrast between both benchmarks. Indeed, for the domestic benchmark, the alphas were significantly negative during the crisis periods, reaching -295.33 basis points per year, but became neutral during the non-crisis periods. For the European benchmark, the alphas were significantly negative during the non-crisis periods, varying between -142.11 and -167.54 basis points per year, but were neutral during the market crises in all the multifactor models used.

This prevalence of significantly negative benchmark alphas in the sampled period aligns with the results of . However, while these authors found that the performance of the domestic benchmark decreases during non-crisis periods, we have only observed the pattern occurring for the European (as opposed to the domestic) index. Hence, the evidence of fund underperformance may lie in model specification errors, which lead to negative reference alphas.

4.2 Unadjusted fund alphas

Table 4 shows the performance (monthly and yearly unadjusted alphas) and risk estimates for equally weighted portfolios of the domestic and European funds in the period 2000-2020 using standard / unadjusted performance evaluation models. We used three alternative models: the 6-factor model (FF6) of equation [2]; a 5-factor variant (FF5) that omits the momentum factor; a four-factor version (CH4) that excludes investment and profitability factors. Besides the total sample period, the results are also shown for the crisis and non-crisis phases separately, following equation [5].

Table 4Unadjusted Fund Performance and Risk Estimates

			Alpha per month (%)	Alpha per year (bps)	Market beta	SMB	HML	RMW	CMA	WML	Adj. R²
Domestic funds	CH4	Total sample	-0.0466	-55.78	0.9952***	0.1834***	-0.0115			-0.1281***	0.9174
			(0.6595)		(0.0000)	(0.0000)	(0.7230)			(0.0006)
		Non-crisis	-0.0436	-52.19	0.9946***	0.1811***	-0.00071			-0.0761	0.9182
			(0.7449)		(0.0000)	(0.0001)	(0.8956)			(0.2301)
		Crisis	0.0586	70.55	0.9892***	0.1885***	-0.0071			-0.1837***	0.9182
			(0.7563)		(0.0000)	(0.0000)	(0.8505)			(0.0000)
European funds	CH4	Total sample	-0.4721***	-552.04	0.9976***	0.1476**	0.0640			-0.0137	0.9113
			(0.0000)		(0.0000)	(0.0437)	(0.2119)			(0.7565)
		Non-crisis	-0.6095***	-707.37	1.0446***	0.2150**	0.1191*			0.0458	0.9131
			(0.0000)		(0.0000)	(0.0162)	(0.0915)			(0.3479)
		Crisis	-0.4292	-503.05	0.9264***	0.0829	0.0832			-0.1040*	0.9131
			(0.2060)		(0.0000)	(0.4833)	(0.1840)			(0.0766)
	FF5	Total sample	-0.3947***	-463.49	0.9621***	0.1241**	0.0928	-0.1967**	-0.1361		0.9152
			(0.0000)		(0.0000)	(0.0480)	(0.1800)	(0.0134)	(0.2239)
		Non-crisis	-0.4679***	-547.25	0.9888***	0.1377*	0.0833	-0.2612***	-0.2652*		0.9157
			(0.0000)		(0.0000)	(0.0646)	(0.3903)	(0.0059)	(0.0761)
		Crisis	-0.3816	-448.43	0.9420***	0.0783	0.0669	-0.2436*	-0.0404		0.9157
			(0.2449)		(0.0000)	(0.5229)	(0.5647)	(0.0765)	(0.8131)
	FF6	Total sample	-0.4078***	-478.53	0.9688***	0.1120*	0.1076*	-0.2210***	-0.1572	0.0304	0.9152
			(0.0000)		(0.0000)	(0.0646)	(0.0967)	(0.0044)	(0.1907)	(0.5029)
		Non-crisis	-0.5060***	-590.58	0.9998***	0.1254*	0.1454	-0.2691**	-0.2966**	0.0634	0.9165
			(0.0000)		(0.0000)	(0.0921)	(0.1114)	(0.0175)	(0.0343)	(0.2178)
		Crisis	-0.3868	-454.41	0.9202***	0.0913	0.0708	-0.1796	-0.0154	-0.0664	0.9165
			(0.3162)		(0.0000)	(0.4702)	(0.4988)	(0.2710)	(0.9309)	(0.3749)

Note: Standard errors were adjusted for autocorrelation and heteroskedasticity (Newey & West, 1987). The P-values are in parentheses.

***,**,* indicate the statistically significant coefficients at the 1%, 5%, and 10% levels, respectively.

The results show that the model has high explanatory power, with values of approximately 92% in both portfolios, which indicates that the right choice was made for the benchmarks. For the total period under evaluation, the unadjusted alpha of the domestic portfolio was negative (-55.78 basis points per year) but not statistically significant, indicating a neutral performance, in line with , but unlike both and , who found negative and statistically significant unadjusted alphas for domestic funds domiciled in the US and UK markets. In contrast, the European funds exhibited significantly negative alphas in all cases, consistent with , with values ranging between -463.49 basis points and -552.04 basis points per year. Additionally, in terms of investment styles, both fund portfolios exhibited significant exposure to small caps, but no clear value or growth tilts. The coefficient of the momentum factor was negative and statistically significant only for the domestic funds, suggesting contrarian strategies. For the European funds, there was also significant exposure to firms with low profitability.

Upon reviewing the alphas obtained for the crisis and non-crisis phases separately, we can see that, for both fund categories, unadjusted alphas were higher for the market crisis phases than for the market non-crisis phases, contrary to . However, while the domestic funds exhibited neutral performance during the two market phases, the European funds showed neutral performance for market crises and significant underperformance for the non-crisis phases. Consequently, the results for the European funds may be due to the lower benchmark alphas during the non-crisis phases, which underestimated fund alphas in the standard models. For the domestic funds, benchmark alphas were significantly lower during the crises, meaning that fund performance may be undervalued, especially during more turbulent times.

The factor coefficients of the domestic funds are similar for the market crisis and non-crisis phases, except for the significantly negative coefficient of the momentum factor, which was only found during the crises. For the European funds, we noted higher market betas, as well as higher exposure to small caps, weaker profitability and higher investment firms, for non-crisis phases than for crisis phases.

4.3 Benchmark-adjusted fund alphas

Table 5 shows performance estimates (yearly alphas, in basis points) before and after the benchmark adjustment procedure for equally weighted portfolios of domestic and European funds for the period 2000-2020. The benchmark-adjusted alphas were based on three model specifications: the 6-factor model (FF6) of equation [3]; a 5-factor variant (FF5) that omits the momentum factor; a four-factor version (CH4) that excludes the investment and profitability factors. The excess betas for each factor can also be seen. The unadjusted alphas were based on equation [2] and its variants. Besides the total sample period, the results are also shown for the crisis and non-crisis phases separately, following equation [6].

Table 5Benchmark-Adjusted Fund Alphas

			Alpha before adj. (bps per year)	Alpha after adj. (bps per year)	Difference (bps per year)	Excess market beta	Excess SMB	Excess HML	Excess RMW	Excess CMA	Excess WML	Adj. R² (within)
Domestic funds	CH4	Total sample	-55.78	88.92 (0.4993)	144.70***	-0.0447 (0.1250)	0.1755*** (0.0000)	-0.0028 (0.9279)			-0.0642** (0.0202)	0.2035
			(0.6595)
						-
		Non-crisis	-52.19	76.59 (0.5843)	128.78***	-0.0641** (0.0402)	0.1640*** (0.0000)	0.0289 (0.4566)			-0.0144 (0.6569)	0.2332
			(0.7449)
		Crisis	70.55	376.05 (0.1505)	305.50***	-0.0283 (0.5159)	0.2029*** (0.0000)	-0.0278 (0.5123)			-0.1159*** (0.0044)	0.2332
			(0.7563)
European funds	CH4		-	-
		Total sample	-552.04***	-445.55*** (0.0000)	106.49***	-0.0030 (0.9048)	0.3632*** (0.0000)	0.0072 (0.8740)			-0.0116 (0.7522)	0.1758
			(0.0000)
			-	-
		Non-crisis	-707.37***	-572.75*** (0.0000)	134.62***	0.0419 (0.1510)	0.4170*** (0.0000)	0.0441 (0.4438)			0.0454 (0.2498)	0.1970
			(0.0000)
		Crisis	-503.05	-485.76 (0.2865)	17.30	-0.0856* (0.0821)	0.3017** (0.0171)	0.0508 (0.5103)			-0.1158* (0.0961)	0.1970
			(0.2060)
	FF5		-	-		-
		Total sample	-463.49***	-343.08*** (0.0002)	120.41***	-0.0426** (0.0358)	0.3398*** (0.0000)	0.0322 (0.5509)	-0.2213*** (0.0002)	-0.1411 (0.1610)		0.2257
			(0.0000)
			-	-
		Non-crisis	-547.25***	-385.65*** (0.0001)	161.60***	-0.0225 (0.3949)	0.3236*** (0.0000)	0.0084 (0.8942)	-0.3127*** (0.0000)	-0.3171***		0.2442
			(0.0000)							0.0169
		Crisis	-448.43	-415.70 (0.1983)	32.73	-0.0665 (0.1215)	0.3066** (0.0163)	0.0108 (0.9248)	-0.3003*** (0.0037)	-0.0145 (0.9074)		0.2442
			(0.2449)
	FF6		-	-
		Total sample	-478.53***	-361.78*** (0.0001)	116.75***	-0.0343 (0.1156)	0.3249*** (0.0000)	0.0503 (0.3585)	-0.2511*** (0.0000)	-0.1671 (0.1278)	0.0374 (0.3398)	0.2284
			(0.0000)
			-	-
		Non-crisis	-590.58***	-431.85*** (0.0000)	158.73***	-0.0110 (0.6387)	0.3108*** (0.0000)	0.0736 (0.2108)	-0.3210*** (0.0000)	-0.3500*** (0.0047)	0.0665* (0.0988)	0.2554
			(0.0000)
		Crisis	-454.41	-421.93 (0.2143)	32.48	-0.0896** (0.0495)	0.3203*** (0.0056)	0.0149 (0.8930)	-0.2325* (0.0648)	0.0119 (0.9277)	-0.0703 (0.2753)	0.2554
			(0.3162)

Note: Standard errors were adjusted for autocorrelation and heteroskedasticity (Newey & West, 1987). The P-values are in parentheses.

***,**,* indicate the statistically significant coefficients at the 1%, 5%, and 10% levels, respectively.

The significance of the difference between the unadjusted and adjusted alphas was determined by a Z-Test = $\frac{α_{b e f o r e} - α_{a f t e r}}{\sqrt{(SE α_{b e f o r e})^{2} +(SE α_{a f t e r})^{2}}}$

For the full sample period, we found a statistically significant increase in the alpha estimates (at the 1% level) when switching from unadjusted to benchmark-adjusted alphas, for both fund categories and regardless of the performance evaluation model applied. Although we have evidence of neutral performance for the domestic funds, the alphas increased by 144.70 basis points, soaring from -55.78 to 88.92 basis points per year. This increase of, approximately, 145 basis points per year was considerably higher than the 86 basis points registered by for US funds and the 127 basis points reported by for UK funds. Therefore, the difference between pre- and post-adjustment alphas seems to be higher for funds domiciled in smaller markets. The European funds still exhibited significantly negative alphas, but these increased by between 106.49 and 120.41 basis points per year. This means that the differences between unadjusted and benchmark-adjusted alphas were higher for the funds investing in local stocks than for the ones investing abroad.

With reference to the excess betas, we can see that both the domestic and European funds were significantly more exposed to small caps than the PSI or the FTSE Eurofirst indices, in line with Table 4. Additionally, compared to their benchmarks, the domestic funds show significantly lower loadings on the momentum factor, while the European ones exhibited significantly lower exposure to the profitability factor.

When comparing the results between the crisis and non-crisis phases, we uncovered some interesting results, especially concerning the contrast between fund categories. For the non-crisis periods, for both the domestic and the European funds, we can observe significant differences at the 1% level between pre- and post-adjustment alphas. In this case, the benchmark-adjusted alphas were significantly higher than the unadjusted alphas by 128.78 basis points per year for the domestic portfolio and between 134.62 and 161.60 basis points per year for the European portfolio. The higher differences for the European funds may be because only the European benchmark displayed a significantly negative alpha during the non-crisis phases, while the domestic benchmark exhibited a neutral alpha, thereby underestimating the unadjusted alphas by a greater margin. As a result, while the European funds exhibited higher differences in the alphas for the non-crisis phases than for the overall period, for the domestic funds, we found the opposite to be true.

During the crises, the benchmark-adjusted alpha of the domestic fund portfolio was significantly higher than the unadjusted alpha, at the 1% level, with the difference reaching a noteworthy 305.50 basis points per year. This result is undeniably related to the considerably lower (and statistically significant) benchmark alphas obtained for the domestic benchmark during the crisis periods. Conversely, for the European funds, there were no significant differences between unadjusted or adjusted alphas during the market crises in any of the model specifications used, in line with the neutral benchmark alphas obtained during these periods.

5. CONCLUSIONS

This work analyzes whether the use of benchmark-adjusted alphas, as opposed to standard (non-adjusted) alphas, leads to different inferences in terms of mutual fund performance. Since previous evidence on this research topic has been restricted to large mutual fund markets, such as those of the USA and the UK, and to funds investing mostly in local securities, we have focused our analysis on a dataset of mutual funds based in a small market (Portugal), investing both in domestic and in European equities, over a 21-year evaluation period, between January 2000 and December 2020. Besides this, we have split and compared both benchmark and mutual fund performance in market crisis and non-crisis phases.

For the entire duration of the sample, our results show that both benchmark indices exhibited negative and statistically significant alphas, meaning that mutual fund alphas have a downward bias if performance is evaluated using standard models. The unadjusted alphas were neutral for the funds investing in their domestic market and significantly negative for the ones investing in the European market. Therefore, fund managers were unable to beat the market, consistent with the majority of prior research on mutual fund performance. However, the benchmark adjustment procedure increased fund alphas significantly. Indeed, although the domestic funds remained neutral performers and the European funds still significantly underperformed, the benchmark-adjusted alphas were significantly higher than the standard alphas for both fund categories, at the 1% level. The differences between the pre- and post-adjustment alphas reached 145 basis points per year and were considerably higher than the ones reported for larger fund markets, with less concentrated benchmarks, such as those of the USA and the UK. Furthermore, they were also higher for the funds investing in local stocks than for the ones investing abroad.

The results obtained separately in the market crisis and non-crisis phases, highlight several interesting differences between both benchmarks and fund categories. While the domestic benchmark exhibited neutral alphas in the non-crisis phases and significantly negative alphas in the market crises, for the European benchmark we found the opposite to be the case. However, the unadjusted alphas were higher for the market crises than for the non-crisis phases, for both fund categories, in clear contrast to . Likewise, the domestic funds displayed an identical (neutral) performance for both market phases, while the European funds showed neutral performance for the market crises and significant underperformance for the non-crisis periods.

During the non-crisis phases, the difference between the pre- and post-adjustment alphas were statistically significant for both the domestic and European funds but were higher for the latter (reaching 162 basis points per year) than for the former (129 basis points per year). This result may be related to the lower (and significantly negative) alpha of the European benchmark in those periods, which would have led to a higher underestimation of the standard alphas. On the other hand, during market crises, benchmark-adjusted alphas were significantly (at the 1% level) higher than the unadjusted alphas for the domestic funds only, with the differences reaching 306 basis points per year. A probable justification for this finding is the considerably lower (and statistically significant) benchmark alpha of the domestic benchmark in the crisis periods.

Thus, our results show that the generalized mutual fund underperformance may, at least partially, have been caused by negative benchmark alphas; additionally, the benchmark-adjustment procedure may have had a higher impact when the benchmark indices exhibited higher concentration. Therefore, the use of benchmark-adjusted alphas is crucial for an accurate assessment of mutual fund performance and should contribute to a more optimistic view of the value of active fund management.

This study has several practical implications for evaluating and understanding mutual fund performance in small markets. Firstly, fund managers and investors should consider using benchmark-adjusted alphas to obtain a more reliable assessment of fund performance, particularly during periods of market instability and in small markets with concentrated benchmarks. Secondly, it is important to carefully select benchmarks that accurately reflect the characteristics and investment strategies of the funds studied. By recognizing the impact of benchmarks and employing suitable adjustment procedures, fund managers and investors should be able to make more informed decisions, potentially enhancing investment outcomes. Extending this work to other mutual fund markets, especially small or emerging markets, would be appealing for future research. Studying the effects of persistence, market timing, and selectivity that arise from accepting or rejecting benchmark-adjusted alphas would be another recommended future line of research.

Author contribution

Conceptualization: FL, PL; Data curation: FL; Formal analysis: FL, PL, MCC; Methodology: FL, PL; Software: FL, PL, MCC; Validation: FL, PL, MCC, PDS; Visualization: FL, MCC, PDS; Writing – original draft: FL, PL, MCC; Writing – review & editing: PL, MCC, PDS.

Acknowledgements

Paulo Leite is grateful to the Foundation for Science and Technology (FCT, Portugal) for financial support by national funds FCT/MCTES to UNIAG (UIDB/04752/2020 and UIDP/04752/2020).

Pablo Durán acknowledges the funding of the Galician Regional Government [ED431C 2020/18] co-funded by the European Regional Development Fund (ERDF/FEDER) within the period 2020–2023.3

6. REFERENCES

Alves, C., & Mendes, V. (2007). Are mutual fund investors in jail? Applied Financial Economics, 17(16), 1-12. http://dx.doi.org/10.1080/09603100600970073

Angelidis, T., Giamouridis, D., & Tessaromatis, N. (2013). Revisiting mutual fund performance evaluation. Journal of Banking & Finance, 37(5), 1759-1776. https://doi.org/10.1016/j.jbankfin.2013.01.006

Bauer, R., Otten, R., & Rad, A.T. (2006). New Zealand mutual funds: measuring performance and persistence in performance. Accounting and Finance, 46(3), 347-363. https://doi.org/10.1111/j.1467-629X.2006.00171.x

Carhart, M. (1997). On Persistence in Mutual Fund Performance. Journal of Finance, 52(1), 57-82. https://doi.org/10.1111/j.1540-6261.1997.tb03808.x

Chinthalapati, V., Mateus, C., & Todorovic, N. (2017). Alphas in disguise: A new approach to uncovering them. International Journal of Finance & Economics, 22(5), 234-243. https://doi.org/10.1002/ijfe.1581

Cremers, M., Petajisto, A., & Zitzewitz, E. (2013). Should benchmark indices have alpha? Revisiting performance evaluation). Critical Finance Review, 2(1), 1-48. http://dx.doi.org/10.1561/104.00000007

Cuthbertson, K., Nitzsche, D., & O'Sullivan, N. (2022). Mutual fund performance persistence: Factor models and portfolio size. International Review of Financial Analysis, 81, 102133. https://doi.org/10.1016/j.irfa.2022.102133

European Fund and Asset Management Association (EFAMA). (2021). Asset Management

Report 2020. Retrieved from https://www.efama.org/newsroom/news/efama-asset-management-report-2022, accessed 06 March 2023.

Fama, E., & French, K. (1993). Common risk factors in the returns on stocks and bonds. Journal of Financial Economics, 33(1), 3-56. https://doi.org/10.1016/0304-405X(93)90023-5

Fama, E., & French, K. (2015). A five-factor asset pricing model. Journal of Financial Economics, 116(1), 1-22. https://doi.org/10.1016/j.jfineco.2014.10.010

Fama, E., & French, K. (2018). Choosing factors. Journal of Financial Economics, 128(2), 234-252. https://doi.org/10.1016/j.jfineco.2018.02.012

Jensen, M. (1968). The Performance of Mutual Funds in the Period 1945-1964. Journal of Finance, 23(2), 389-416. https://doi.org/10.1111/j.1540-6261.1968.tb00815.x

Leite, P., & Cortez, M.C. (2009). Conditioning information in mutual fund performance evaluation: Portuguese evidence. European Journal of Finance, 15(5-6), 585-605. https://doi.org/10.1080/13518470802697378

Leite, P., & Cortez, M.C. (2015). Performance of European Socially Responsible Funds during Market Crises: Evidence from France. International Review of Financial Analysis, 40, 132-141. https://doi.org/10.1016/j.irfa.2015.05.012

Leite, P., & Cortez, M. (2020). Investment and profitability factors in mutual fund performance evaluation: a conditional approach. Applied Economics Letters, 27(16), 1312-1315. https://doi.org/10.1080/13504851.2019.1678723

Mateus, I., Mateus, C., & Todorovic, N. (2016). UK equity mutual fund alphas make a comeback. International Review of Financial Analysis, 44, 98-110. https://doi.org/10.1016/j.irfa.2016.01.004

Mateus, I., Mateus, C., & Todorovic, N. (2019a). Benchmark-adjusted performance of US equity mutual funds and the issue of prospectus benchmarks. Journal of Asset Management, 20, 15-30. https://doi.org/10.1057/s41260-018-0101-z

Mateus, I. B., Mateus, C., & Todorovic, N. (2019b). Review of new trends in the literature on factor models and mutual fund performance. International Review of Financial Analysis, 63, 344-354. https://doi.org/10.1016/j.irfa.2018.12.012

Neto, N., Lobão, J., & Vieira, E. (2017). Do Portuguese mutual funds display forecasting skills? A study on selectivity and market timing ability. Studies in Economics and Finance, 34(4), 597-631. https://doi.org/10.1108/SEF-09-2015-0233

Pagan, A., & Sossounov, K. (2003). A simple framework for analysing bull and bear markets. Journal of Applied Econometrics, 18(1), 23-46. https://doi.org/10.1002/jae.664

Romacho, J., & Cortez, M.C. (2006). Timing and selectivity in Portuguese mutual fund performance. Research in International Business and Finance, 20(3), 348-368. https://doi.org/10.1016/j.ribaf.2005.05.005

Sharpe, W. (1966). Mutual Fund Performance. Journal of Business, 39, 119-138. http://dx.doi.org/10.1086/294846

Treynor, J. (1965). How to rate Management of Investments Funds. Harvard Business Review, 43(1), 63-75. ISSN: 0017-8012

Wang, W., Su, C., & Duxbury, D. (2022). The conditional impact of investor sentiment in global stock markets: A two-channel examination. Journal of Banking & Finance, 138(C), 106458. https://doi.org/10.1016/j.jbankfin.2022.106458

Xu, Y., Liang, C., & Wang, J. (2023). Financial stress and returns predictability: Fresh evidence from China. Pacific-Basin Finance Journal, 78, 101980. https://doi.org/10.1016/j.pacfin.2023.101980

Notes

[1] By the end of 2020, the Portuguese investment fund market had €17 billion of assets under management ().

[2] The main Portuguese Stock Index, the PSI, had only 18 stocks at the end of 2020. The weight of the top 5 constituents is usually close to 60%, which is considerably higher than that of indices such as the S&P 500 and the FTSE 100.

[3] Due to the unavailability of the investment and profitability factors for the Portuguese market, 5- and 6-factor versions can only be applied to the European funds category. For domestic funds, we have used the 4-factor specification.

[4] To estimate 5-factor (unadjusted and benchmark-adjusted) alphas we have omitted the factor from equations [2] and [3], respectively. 4-factor alphas are obtained by excluding and factors.

[5] Again, for robustness purposes, we have estimated 4- and 5-factor versions of these models.

[6] European equity funds are included because, throughout our evaluation period, they were the most representative international fund category in the market.

[7] In comparison to European funds, domestic funds are, on average, larger (€17.34 vs. €13.12 million) and slightly older (17.08 vs. 15.21 years), but expense ratios are very similar for both categories (1.88% vs. 1.83% per year).

[8] This data is freely available at https://web3.cmvm.pt/english/sdi/fundos/app/index.cfm.

[9] For the period under analysis (2000-2020), correlations between the two indices were reasonably high, reaching 72.29%.

[10] http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/index.html

[11] https://www.aqr.com/Insights/Datasets/Betting-Against-Beta-Equity-Factors-Monthly

[12] At this stage, to facilitate comparisons and allow a clearer interpretation of Figure 1, we opted for just one model specification for both benchmarks.

[13] In line with , the alphas per month and per year (in basis points) are shown.

[14] As an additional robustness test, we evaluated the impact of fees on performance by computing benchmark-adjusted alphas using gross returns. As we can see in Appendix 1, on a before-fee basis, the domestic funds significantly outperformed the market at the 5% level, reaching an adjusted alpha of 280.13 basis points per year. Accordingly, the neutral performance of domestic funds was justified by the fees charged. In contrast, the European funds significantly underperformed, scoring between -164.35 and -268.56 basis points per year, even before management fees were deducted from fund returns. We thank an anonymous referee for suggesting these tests to us.

[15] In Appendix 1, we can see that, using gross returns, the benchmark-adjusted alpha of the domestic funds was 572.24 basis points per year. This value is not only statistically significant (5% level) but economically relevant. In contrast, the European funds exhibited significantly negative gross alphas during the non-crisis periods and neutral gross alphas during the crises.

Appendix

Appendix 1. Fund Fees and Benchmark-Adjusted Alphas

This appendix presents the benchmark-adjusted net (after-fee) and gross (before-fee) alphas, expressed in basis points per year, for equally weighted portfolios of domestic and European funds, over the 2000-2020 period. Benchmark-adjusted alphas are based on three model specifications: the 6-factor model (FF6) of equation [3]; a 5-factor variant (FF5) that omits the momentum factor; a four-factor version (CH4) that excludes the investment and profitability factors. Besides the total sample period, results are also presented for crisis and non-crisis phases separately, following equation [6].

			Net alpha (bps per year)	Gross alpha (bps per year)
Domestic funds	CH4	Total sample	88.92	280.13**
			(0.4993)	(0.0356)
		Non-crisis	76.59	267.59*
			(0.5843)	(0.0588)
		Crisis	376.05	572.24**
			(0.1505)	(0.0305)
European funds	CH4	Total sample	-445.55***	-268.56**
			(0.0000)	(0.0112)
		Non-crisis	-572.75***	-397.91***
			(0.0000)	(0.0002)
		Crisis	-485.76	-309.44
			(0.2865)	(0.5004)
	FF5	Total sample	-343.08***	-164.35*
			(0.0002)	(0.0689)
		Non-crisis	-385.65***	-207.64**
			(0.0001)	(0.0355)
		Crisis	-415.70	-238.20
			(0.1983)	(0.4643)
	FF6	Total sample	-361.78***	-183.36**
			(0.0001)	(0.0474)
		Non-crisis	-431.85***	-254.62***
			(0.0000)	(0.0058)
		Crisis	-421.93	-244.54
			(0.2143)	(0.4749)

Note: Standard errors were adjusted for autocorrelation and heteroskedasticity (Newey & West, 1987). The P-values are in parentheses

***,**,* indicate the statistically significant coefficients at the 1%, 5%, and 10% levels, respectively