The Art of Encoding Categorical Variables in an Automated Valuation Model (AVM)

Target Audience: New Graduates/Analysts

One-hot coding and effect coding are two common methods used to encode categorical variables in statistical analysis. One-hot coding involves creating binary columns for each category in the variable. Each column represents a category, and a value of 1 indicates the presence of that category, while 0 indicates the absence. This method results in a binary representation of the categories. Effect coding is a contrast coding method that compares each level of a categorical variable to the grand median (in AVM) of all levels. It uses one less variable than one-hot coding and allows for comparison between categories.

Example 1

Let's say we have a categorical variable, "Neighborhood," that describes three neighborhoods: "Downtown," "Suburb," and "Rural" in a city. We want to encode it using one-hot coding and effect coding.

One-Hot Coding:

·         "Downtown" would be encoded as [1, 0, 0]

·         "Suburb" would be encoded as [0, 1, 0]

·         "Rural" would be encoded as [0, 0, 1]

Effect Coding:

·         "Downtown" would be encoded as [-2, 1, 1]

• "Suburb" would be encoded as [1, -2, 1]
• "Rural" would be encoded as [1, 1, -2]

In effect coding, the coefficients sum to zero, with one level as the reference category (-2) and the other as comparisons (1).

In summary, one-hot coding creates binary columns for each category, while effect coding compares each category to the grand median. Both methods have their uses in different statistical analyses and modeling techniques.

While one-hot coding and effect coding are two common methods used to encode categorical variables in statistical analysis, they are not the only methods available. Here are a few other common methods:

1.    Label Encoding: Label encoding assigns a unique numerical value to each category in the variable. This method is simple and efficient but may not be suitable for models that assume ordinal relationships between categories.

2.    Binary Encoding: Binary encoding converts each category into binary digits and represents them in a new set of columns. This method reduces the number of columns compared to one-hot encoding while preserving the information about the categories.

3.    Helmert Coding: Helmert coding compares each categorical variable level with the subsequent levels' median. This method is useful for capturing changes or trends in the categories.

4.    Sum Coding: Sum coding compares each level of a categorical variable with the overall median of all levels. This method provides information about differences between each level and the overall median.

Each encoding method has its advantages and drawbacks, and the choice of method depends on the nature of the categorical variable and the specific requirements of the analysis or model being used.

Continuing with our "Neighborhood" categorical variable in the housing market for the other encoding methods:

1.    Label Encoding: If we use label encoding for the "Neighborhood" variable:

• "Downtown" could be encoded as 1
• "Suburb" could be encoded as 2
• "Rural" could be encoded as 3

2.    Binary Encoding: If we use binary encoding for the "Neighborhood" variable:

• "Downtown" could be encoded as [0, 0]
• "Suburb" could be encoded as [0, 1]
• "Rural" could be encoded as [1, 0]

3.    Helmert Coding: If we use Helmert coding for the "Neighborhood" variable:

• "Downtown" would be encoded as [1, -1, 0]
• "Suburb" would be encoded as [1, 1, -2]
• "Rural" would be encoded as [1, 1, 1]

4.    Sum Coding: If we use sum coding for the "Neighborhood" variable:

• "Downtown" would be encoded as [1, -1, 0]
• "Suburb" would be encoded as [0, 1, -1]
• "Rural" would be encoded as [0, 0, 1]

These examples demonstrate how different encoding methods can represent the same categorical variable in various numerical formats, each with its own characteristics and implications for analysis or modeling in the housing market context.

One-Hot Coding

When dealing with a categorical variable with more than two levels (e.g., four categories: Mid-density Non-HOA, High-density Non-HOA, PUD, and MPUD), AVM analysts can use a technique called one-hot encoding to represent these categories in a regression model.

One-hot encoding involves creating binary dummy variables for each category of the categorical variable. Each dummy variable represents one category and takes on values of 0 or 1.

Here's how they can encode the above categorical variable with four categories for a regression model:

1.    Mid-density Non-HOA: This category would be represented by a dummy variable that takes on the value of 1 if the observation falls into this category and 0 otherwise.

2.    High-density Non-HOA: Another dummy variable would be created for this category with the same logic as above.

3.    PUD: They would assign a dummy variable for this category.

4.    MPUD: Similarly, they would create a dummy variable for this category.

Using one-hot encoding in this manner, they can incorporate a categorical variable with multiple levels into their regression model, allowing them to analyze the impact of different categories on the dependent variable while maintaining the regression model's linearity assumption.

Important Note

AVM analysts will create four dummy variables to represent each category using one-hot encoding for a categorical variable with four distinct categories (Mid-density Non-HOA, High-density Non-HOA, PUD, and MPUD). Each dummy variable would take on 1 if the observation belongs to that particular category and 0 if it does not. This approach allows them to include all four variable categories in the regression model and analyze their effects on the dependent variable separately.

Effect Coding

Example 2

Effect coding a categorical variable like the TOWNSHIP variable in a regression model can be statistically valid and useful for comparing the mean (or median) response across different categorical variable levels to an overall mean (or median).

By effect coding the TOWNSHIP variable, the model examines how each township's median SP/SF deviates from the countywide median. This approach allows for the evaluation of the average effect of each township on the dependent variable, Adjusted Sale Price while accounting for the countywide effect.

Median vs. Median in Effect Coding

Even if the data series does not contain many outliers, using the median instead of the median can still be preferable in certain instances, especially in an AVM. The median is a robust measure of central tendency less affected by extreme values than the median. By using the median to code a categorical variable like the TOWNSHIP variable in a regression model, analysts ensure that the central tendency measure chosen is not unduly influenced by any outliers in the data.

In cases where the data is skewed or if there are concerns about outliers impacting the average, the median can provide a more stable estimate of the central tendency and better reflect the typical value within each township. Therefore, even if the data series does not contain many outliers, selecting the median over the median can still be suitable for creating more robust and reliable effect coding for the regression model.

Case for Effect Coding

Effect coding can be employed to represent a categorical variable with four levels (Mid-density Non-HOA, High-density Non-HOA, PUD, and MPUD) as a single variable in a regression model for sale price while preserving linearity. This technique captures the combined influence of all four categories on the dependent variable by considering their deviations from the overall median. By incorporating these effect-coded variables into the model, analysts can analyze the impact of the categorical variable while maintaining a linear relationship with the sale price.

Example 3

Let's consider a numerical example to demonstrate effect coding for a categorical variable with four categories: Mid-density Non-HOA, High-density Non-HOA, PUD, and MPUD, with the following data:

·         Mid-density Non-HOA: Median Sale Price = $300,000

·         High-density Non-HOA: Median Sale Price = $320,000

·         PUD: Median Sale Price = $350,000

·         MPUD: Median Sale Price = $380,000

·         Overall Median Sale Price = $337,500

To apply effect coding, the differences between each category median and the overall median have to be calculated:

1.    Mid-density Non-HOA Effect: $300,000 (category median) - $337,500 (overall median) = -$37,500

2.    High-density Non-HOA Effect: $320,000 (category median) - $337,500 (overall median) = -$17,500

3.    PUD Effect: $350,000 (category median) - $337,500 (overall median) = $12,500

4.    MPUD Effect: $380,000 (category median) - $337,500 (overall median) = $42,500

Effect-coded variables for the four categories using these calculated values have been created. For each observation, the effect-coded variable will reflect the difference between the category median and the overall median of Sale Price. In the regression model, analysts would include these effect-coded variables and other independent variables to analyze the impact of the categorical variable on the dependent variable (Adjusted Sale Price) while maintaining linearity.

Important to Note

Creating the effect-coded variable using a normalized variable such as SP/SF instead of the sale price itself can be a reasonable approach, especially if analysts are interested in capturing the impact of the categorical variable on the price per square foot specifically.

When using normalized variables, they are standardizing the metric or unit of measurement, which can help improve the interpretability and comparability of coefficients in the regression model.

In this case, creating effect-coded variables based on the SP/SF (a normalized measure) for the four categories can help them assess how each category influences the price per square foot compared to the overall price per square foot median.

However, it is important to keep in mind that when using normalized variables, the interpretation of the coefficients in the regression model will be based on the price per square foot, not the absolute sale price. Additionally, including the living area as a separate independent variable in the regression equation will allow them to analyze its individual effect on the dependent variable while assessing the impact of the categorical variable on the price per square foot.

Conclusion

One-hot and effect coding are valuable techniques for transforming categorical variables into a format usable by automated valuation models (AVMs). Here's a breakdown of their strengths and weaknesses to help you choose the best approach for your AVM:

One-Hot Coding:

·         Strengths: Simple to implement, easy to interpret coefficients, avoid multicollinearity.

·         Weaknesses: Increases model complexity with many categories, can be data-hungry for large datasets.

Effect Coding:

·         Strengths: It is more efficient for models with many categories, reduces multicollinearity, and allows for comparisons between specific categories.

·     Weaknesses: Interpretation of coefficients can be less intuitive and requires choosing a reference category that may influence results.

Choosing the Right Approach:

·         One-hot coding is a good choice for AVMs with a small number of categorical variables or when a clear interpretation of individual category coefficients is crucial.

·         Effect coding is preferable for AVMs with many categories or when you're interested in comparing specific categories (e.g., pool vs. no pool).

The optimal approach depends on the specific characteristics of the AVM and the intended research questions. To determine the most suitable method, it is recommended that the model be run with both coding approaches and that the results be compared. This comparison will reveal which method yields the most interpretable and accurate data assessment.

Sid's AI-Assisted Bookshelf: Elevate Your Personal and Business Potential