Many large municipalities face dire financial
challenges resulting from shrinking commercial tax base due to rising office
vacancies and falling property prices (e.g., some properties have recently sold
for land values, etc.) from the continued work-from-home trends, so they must
not only avoid wasteful expenditures but also explore new and innovative
methods to enhance accuracy and reliability of Computer-Assisted Mass Appraisal
(CAMA) models to implement stable assessment rolls, vastly reducing the incidence
of appeals, refunds, and risks of expensive ratio trials.
In the world of
CAMA, it is commonly understood that the quality of CAMA models should be
assessed by performing a series of error tests on the sales samples used for
modeling. However, these samples are often not subject to thorough testing to
ensure they represent the entire population. As a result, the values obtained
from these samples may need to accurately reflect the modeling statistics'
overall quality. Therefore, evaluating a set of sales ratios from forward sales
samples is essential to ensure the quality and effectiveness of the population
values.
Forward samples
derived from more recent sales and adjusted to the valuation date can
significantly benefit CAMA modeling. If CAMA models based on Multiple
Regression Analysis (MRA) are built efficiently and adhere to proper
econometric requirements, the results from forward samples could closely
resemble those from the initial modeling samples, thus improving model accuracy
and reliability.
Unfortunately,
many jurisdictions in the US and Canada still rely on CAMA packages with
outdated, non-econometric methodologies. These packages often lack the
necessary tests to ensure that modeling sales samples are representative,
leading to a one-size-fits-all approach to modeling. This underscores the
urgent need for improved methodologies and highlights the potential benefits of
forward sample testing.
The need for
forward sample testing arises because many CAMA models are often developed
separately from their populations. In other words, modelers using outdated
platforms may start the modeling process when the sales samples are created
without confirming whether those samples genuinely represent the unsold
populations to which the models will eventually be applied. Even where the
prior sample tests are meaningfully performed, tracking the forward sales
samples and ratios would be a sound statistical practice to identify the
geographic areas or the stretches on the value curve (e.g., the longer end of the
value curve, etc.) where models tend to fail or return less than adequate
results.
Potential Benefits
Forward sales samples can be beneficial for assessing the
effectiveness of CAMA models, particularly in the context of the changing real
estate landscape and the challenges municipalities face. Here are some key benefits
of using forward sales samples in CAM:
Reflecting Current Market Conditions:
Forward sales samples derived from more recent transactions and adjusted to the valuation date can help capture current market conditions more
accurately. This can lead to more reliable and up-to-date review and validation
for CAMA models.
Improving Model Accuracy: By
incorporating forward sales samples into the testing and validation process of
CAMA models, assessment departments can enhance the accuracy and reliability of
their valuation models. This could potentially reduce the incidence of appeals
and refunds, ultimately leading to more stable assessment rolls.
Identifying Model Weaknesses:
Tracking forward sales samples and ratios can help identify areas where CAMA
models may fail or produce less accurate results. This can enable assessment
departments to make targeted improvements to their modeling methodologies and
better address any shortcomings in the valuation process.
Reduced Appeals and Refunds: A more accurate model can lead to fewer
property tax disputes, saving large tax jurisdictions significant time and money.
Risk Mitigation: By identifying potential issues early,
assessment departments can proactively address them, reducing the
risk of expensive ratio trials.
Evolving Methodologies: Given the rapid changes in the
real estate market, it is crucial for assessment departments to continuously
evolve their methodologies and embrace more modern and econometric-based approaches
to CAMA modeling. Incorporating forward sample testing can be a step in the
right direction toward modernizing valuation practices.
In summary, forward sales samples can potentially enhance the effectiveness of CAMA models. Assessment departments should seriously consider incorporating this method into their valuation processes. By
adopting more dynamic and data-driven approaches to assessment, municipalities
can better navigate the financial challenges they face and ensure the accuracy
and reliability of their property valuations.
Linear CAMA Models
Since
most CAMA models are based on linear multiple regression analysis (MRA), they
tend to underperform at both ends of the value curve, i.e., below the 25th and
above the 75th percentile of the curve, generally producing less efficient
values and resulting in mass appeals.
They
consistently overpredict at the shorter end of the curve while grossly under-predicting
at the longer end, causing severe regressive situations that lead to
"cause for concern."
However,
these models generally perform reasonably well in the middle of the curve,
i.e., between the 25th and 75th percentile. Therefore, in addition to the
model-wise sales ratio stats (COD, PRD, etc.), those who practice linear modeling
should also examine those stats separately—below the 25th percentile, 25th to
75th percentile, and above the 75th percentile—to stay alert and cautious about
the potential failures at either end of the curve.
Analysis
Many assessment
departments face the challenge of MRA and CAMA models underperforming at the
ends of the value curve due to the inherent limitations of linear models, which
assume a linear relationship between the independent variables (e.g., property
characteristics) and the dependent variable (property value).
Nonlinear
Relationships: The relationship between property
value and characteristics can often be nonlinear. For example, the value of a
property might increase exponentially or logarithmically with the size of the lot rather than
linearly.
Prevalence of
Outliers: Outliers (extreme values) can
disproportionately influence the model's coefficients, leading to inaccurate
predictions at the extremes.
Data Sparsity: There may be fewer data points at the
extremes of the value curve (significantly below the 10th and above the 90th),
making it more difficult for the model to accurately capture the relationship between variables in those regions.
Model Performance: Linear MRA models often struggle to
accurately predict property values at the extreme ends of the value curve.
Below the 25th percentile and above the 75th percentile, these models tend to
exhibit significant over-prediction and under-prediction, respectively. This
can lead to mass appeals and regressive situations, causing concerns about the
fairness and accuracy of property assessments.
Value Segments: Given the varying performance of CAMA models
across different segments of the value curve, assessment departments must
analyze model-wise sales ratio statistics separately for each segment. By
examining performance metrics such as Coefficient of Dispersion (COD) and Price
Related Differential (PRD) below the 25th percentile, between the 25th and 75th
percentile, and above the 75th percentile, analysts can gain insights into
where the models may be failing and adjust their methodologies accordingly.
Forward Sales
Samples: Forward sales samples can be
invaluable in monitoring these value segments and evaluating the effectiveness
of CAMA models across the entire curve. By incorporating more recent sales data
and adjusting it to the valuation date, analysts can track how well the models
perform in real-time and identify any discrepancies or inaccuracies in their
predictions.
Identifying
Failures: By using forward sales samples to
analyze value segments, assessment departments can proactively identify areas
where the models may be underperforming. This early detection allows analysts
to make targeted adjustments to their modeling techniques, address issues at
the ends of the curve, and improve the overall accuracy of property valuations.
In summary,
monitoring value segments and utilizing forward sales samples can help
assessment departments address the challenges associated with linear MRA
models. This approach ensures more accurate and reliable property assessments and allows significant improvement. By
focusing on the performance of CAMA models at different points along the value
curve and leveraging real-time data from forward samples, analysts can enhance
the quality of their valuation processes and minimize the risks of appeals and
potential regressive situations.
Hybrid CAMA Models
The
forward sample test is paramount, where final values are created by a hybrid
process—top-down MRA-based values and bottom-up comparable sales analyses. Many
jurisdictions worldwide are still half-sold on the mass appraisal concept and
modeling. Because they lack confidence in the MRA process, they try to
supplement it by averaging (watering down) the statistically significant MRA
values with highly subjective comparables (generally three to five comps).
Suppose
the MRA returns a value of $500K when determining the value of parcel X, while
the comps produce $400K. As a result, the final roll value for that parcel
would be $450K. While jurisdictions aim for transparency, explicability,
decomposability, and limited experimentation, they often unknowingly introduce
high subjectivity (and perhaps bias) into the valuation process, resulting in
unsmoothed and jagged values throughout the roll.
In
hybrid environments, where the comps have a constrained bottom-up contribution
to the final values, the overall modeling COD would be significantly lower than
the COD from the MRA model alone. For example, if the overall COD is 9, the
Comps-only COD would be closer to 6, and the MRA-only COD would be around 12.
Although these COD statistics seem favorable initially, the CODs from the
forward samples are likely to be much higher, around 14, as the roll would
already be published by then, and the original comps would be embedded.
It
is crucial to avoid future confusion by only publishing the COD from the MRA
component, as the CODs are only intended for mass appraisal models. This
practice ensures transparency and clarity in the valuation process, as the CODs
from efficient MRA models are comparable across all forward samples.
Continuously
tracking results from the forward sales samples is critical, as it ensures the
reliability and accuracy of the models and allows for the identification and
investigation of any potential issues.
Analysis
The issue of a hybrid approach in mass appraisal
models, combining top-down MRA-based values with bottom-up comparable sales
analyses, is a critical consideration for assessment departments. Here's how
forward sales samples can help monitor value consistency in hybrid CAMA
modeling environments:
Hybrid Approach and Subjectivity: In hybrid modeling environments where MRA values are averaged
with comparable sales analyses, there can be a risk of introducing subjectivity
and bias into the valuation process. This hybrid approach may lead to
unsmoothed and jagged values throughout the assessment roll, potentially
impacting the accuracy and reliability of property valuations.
Impact on Coefficient of Dispersion (COD): Combining MRA values and comparable sales analyses can result in an overall COD lower than what would be obtained from the
MRA model alone. While lower COD statistics may initially seem favorable,
including highly subjective comps can lead to higher COD values when considering
forward samples due to the presence of embedded original comps and potential
inconsistencies in valuation.
Transparency and Clarity: To maintain transparency and clarity in the valuation process, it is crucial to avoid confusion by only publishing the COD derived from the MRA component. This practice ensures that the COD values reported are based on the
statistical efficiency of the MRA models and can be compared consistently
across all forward samples, providing a transparent and standardized measure of
valuation accuracy.
Monitoring Value Consistency: Continuously tracking results from forward sales samples is
essential for monitoring the consistency of values derived from CAMA models comprising
the assessment roll. By analyzing how well the models perform over time and
across different market conditions, assessors can identify any discrepancies or
trends that may indicate the need to adjust the valuation methodology.
Role of Forward Sales Samples: Forward sales samples play a crucial role in helping assessors
monitor the reliability and accuracy of CAMA models. By evaluating the models' performance against more recent sales data, assessors can ensure that the values produced reflect current market conditions and
maintain consistency in property assessments.
In summary, forward sales samples can help monitor
value consistency in CAMA models incorporating a hybrid valuation approach.
By tracking the results from forward samples and focusing on the statistical
efficiency of the MRA component, assessment departments can maintain
transparency, accuracy, and reliability in their property valuations,
ultimately improving the effectiveness of the assessment process.
Sales Chasing
Sales
chasing is prohibited in mass appraisal modeling. If models are built chasing
sales, they could be easily identified. Efficient MRA models generate
market-significant values; therefore, forward samples would demonstrate similar
stats to their modeled counterparts. Alternatively, those models must be
investigated if the forward sales samples show much worse stats (much higher
COD, irrational PRD, etc.).
State
equalization boards must prioritize using forward sales samples to
establish fair and equitable equalization rates. This approach, free from sales
manipulation, will give all stakeholders security and confidence.
Local
Value Adjustment Boards and Review Commissions can also use forward samples to
identify inaccuracies in CAMA models, enabling them to manage resources and
pinpoint higher-risk areas efficiently.
Local
mass appeals filers can also benefit from forward sales samples. These samples
can help them quickly identify model failures and optimize their marketing
strategies.
Given
the dynamic nature of local housing markets, forward samples are crucial in defining divergence points. This information is essential for all
stakeholders, ensuring they are well-informed and prepared for market
changes.
Although
some independent consultants are proposing challenger models to reduce sales
manipulation, this could be costly for financially struggling municipalities.
Instead, promoting forward sales samples can level the playing field and save
costs. IAAO and State Equalization boards should seriously consider and promote
this option to avoid expensive ratio trials, providing reassurance and
confidence to all stakeholders.
Forward Sales Samples to Mitigate
Potential Sales Chasing
Sales chasing is
prohibited in mass appraisal modeling. If models are built chasing sales, they
could be easily identified. Efficient MRA models generate market-significant
values; therefore, forward samples would demonstrate similar stats to their
modeled counterparts. Alternatively, those models must be investigated if the
forward sales samples show much worse stats (much higher COD, irrational PRD,
etc.).
State equalization
boards must prioritize using forward sales samples to establish fair and
equitable equalization rates. Free from sales manipulation, this approach will give all stakeholders security and confidence.
Local Value
Adjustment Boards and Review Commissions can also use forward samples to
identify inaccuracies in CAMA models, enabling them to manage resources and
pinpoint higher-risk areas efficiently.
Local mass appeals
filers can also benefit from forward sales samples. These samples can help them
quickly identify model failures and optimize their marketing strategies.
Given the dynamic
nature of local housing markets, forward samples are crucial in defining
divergence points. This information is essential for all stakeholders, ensuring
they are well-informed and prepared for market changes.
Although some independent consultants are proposing challenger models to
reduce sales manipulation, this could be costly for financially struggling
municipalities. Instead, promoting forward sales samples can level the playing
field and save costs. IAAO and State Equalization boards should seriously consider
and promote this option to avoid expensive ratio trials, providing reassurance
and confidence to all stakeholders.
Conclusion
This blog post highlights the importance of forward sales samples
as a risk management tool, especially given many taxing jurisdictions'
challenging financial circumstances. Here is a summary of the key points
discussed:
1. Financial Challenges:
Many municipalities face dire financial circumstances due to shrinking
commercial tax bases, falling property values, and rising office vacancies
resulting from continued work-from-home trends.
2. Risk Management with Forward Sales
Samples: Forward sales samples are crucial for assessing the
effectiveness of CAMA models, particularly in monitoring the accuracy and
reliability of valuations across different segments of the value curve.
3. Challenges of Linear MRA Models:
Linear multiple regression analysis (MRA) models often underperform at the ends
of the value curve, leading to potential mass appeals and regressive
situations. Forward sales samples can help identify these issues and improve
model accuracy.
4. Hybrid Modeling Approaches:
A hybrid approach combining MRA values with comparable sales analyses can
introduce subjectivity and bias into the valuation process. Monitoring forward
sales samples is essential to ensure consistency and reliability in the
valuation models.
5. Avoiding Sales Chasing:
Sales chasing, where models are built using recent sales data, can lead to
inaccuracies in valuations. Forward sales samples can help minimize sales
chasing and enhance model efficiency, stability, and customer confidence.
6. Promoting Cost-Effective Solutions:
Instead of costly challenger models, promoting forward sales samples can level
the playing field, save costs, and provide reassurance and confidence to
stakeholders.
In conclusion, leveraging forward sales samples as a risk
management tool is essential for taxing jurisdictions to navigate financial
challenges, improve model accuracy, and ensure fair and transparent property
valuations. By monitoring forward samples and proactively addressing issues
identified, assessment departments can enhance the reliability of their
valuation models and build trust among stakeholders.
Sid's Bookshelf: Elevate Your Personal and Business Potential
