Assessing Farmers' Market Models
INDICATORS FOR IMPACT:
Understanding the Influence of Farmer's Markets
Dantrell Cotton, Master's Candidate, Nelson Institute for Environmental Studies
Hannah Fritsch, Undergraduate Student, Agricultural Business Management
Colton Schara, Undergraduate Student
Overview
Literature Review
JeongMorales201502FarmerMarketMetric.doc
The project team reviewed a literature review conducted by Dr. Alfonso Morales and Youn Hee Jeong from the University of Wisconsin- Madison. They conducted a literature review of research studies conducted between 1999-2015. The review was conducted with the idea of markets needing a better way to gather data than by looking at case studies of other markets. The review highlights methods that we used in case studies to gather information, and goes in-depth into the pros and cons of the methods used to evaluate the economic, human, social, and ecological impacts. From the review, we were able to create three hypotheses that we used while conducting the pilot year research.
Our goals and AFRI’s coincide, which is to provide market managers and vendors with a set of tools for understanding their markets, gauging their impact, and decision-making. More specifically, we are interested in evaluating the impacts of AFRI’s data collection metrics on the three pillars of sustainability - economic, social and environmental. To achieve this goal, the research team developed a data interpretation and collection package (DAIP) to measure impacts. The purpose of the DAIP is two-fold: a) to describe the utility of each metric to stakeholders, and b) to provide data for markets to generate short and long-term analysis.
We hypothesize that the DAIP will: 1) help markets understand and apply the data they are collecting through the DCP; 2) allow markets to utilize their data for funding, community engagement and stakeholder analysis. For example, markets can use info to communicate with the community on the direct economic impacts on the local economy; and 3) assist markets in developing short and long-term inferential annotations related to market planning, payment options, stakeholder analysis, consumer surveys, and impact on community health.
ABOUT
What is the Indicators for Impact Project?
Indicators for Impact:Farmers Markets as Leaders in Collaborative Food System Data Collection and Analysis is a research project sponsored by the Agriculture and Food Research Initiative (AFRI) that seeks to conduct a large-scale analysis of the ecological, economic, and environmental impacts of farmer’s markets and communicate this information to the surrounding community and potential donors. Moreover, the project will create a customizable data collection platform specifically for farmer’s markets that will be accessible across the nation for other markets to use, and will generate data that can be used by researchers to run competitive analyses and understand the context of markets.
AFRI was established by Congress in the 2008 Farm Bill, and is "the nation's premier peer reviewed competitive grants program for fundamental and applied agricultural science." AFRI granted the Indicators for Impact project $5 million and the project is currently in its third year of funding. The first year was dedicated to creating surveys, setting up an online portal to collect data, and finding markets to participate; the second year marked the launch of the program, with markets beginning to record data and send it in to the project team. This third year will consist of changes made to the structure of the project after the initial launch, analyzing the data received from the previous summer, and seeking additional funding with increased market participation. Just recently, the project team was able to sign up more markets; an exciting development!
Currently the Indicators for Impact research is hosted at the University of Wisconsin- Madison. The project team is working with University of Wisconsin-Madison students to collect and analyze the data that is recorded by the nine pilot markets. The principle investigator is Alfonso Morales, who is a Professor with the Department of Urban and Regional Planning at the University of Wisconsin – Madison. He has 25 years of experience working with food systems, public marketplaces, and street vendors, and the role and function that they serve in economic development. He is well-published, with four books, and more than 40 articles and book chapters. One of the project co-investigators is Jen Cheek of the Farmers Market Coalition, and the other is Lauren Suerth, who is working towards her PhD in Urban and Regional Planning at the University of Wisconsin - Madison. Before joining the Indicators for Impact team, Lauren worked in the private sector as a Risk Analyst in Denver, Colorado. In addition to the nine national markets in the project, Suerth and Morales are working with a number of Wisconsin Main Street markets to collect specific data within the state. They are working to answer the question: how can market managers benefit from an online data collection process and portal?
How is data collected?
Markets and those organizations that host or support markets vary in the kinds of information they need and our data collection process is responsive to those circumstances. The process engages managers in determining their chief goals, operationalizes those by selecting indicators and metrics, and collects data on those to reveal progress towards goals. The overall process proceeds in two steps. First, each market executes a profile for their market that describes a number of the relatively stable features of the market, for instance the address of the market, the business model (nonprofit, city administered etc), how the market is managed, the labor involved, and the number of employees, how visitors arrive to the market (e.g., biking, bus, walking, etc.), how customers can purchase products, (e.g. SNAP, Senior FMNP, etc.), as well as other questions. Additionally, annual reports can be uploaded to help the research team become familiar with the market or organization. The market profile is updated once each year. Many of these questions are of interest, for instance the age and ethnicity/gender/race of the farmer vendors who sell at the market, and an ecologically important variable is how the farmers are producing their products. The collection of this information before the survey process begins allows the project team to have an overall picture of the market, and can be used to categorize markets for better comparison in the future.The second part of the process is collecting data on the performance measures each market selects. This may include information about market sales, visitor counts, how much money visitors spent at neighboring businesses, and 25 other performance measures that can be collected in a multitude of ways.
Each performance measure, called metrics, falls into one of three categories of collection; observations, surveys, and operations research. Within the three collection categories, the metrics are correlated to instruments that were used to collect data (Table 1). There are four required metrics that every market must report on; the markets then choose any additional metrics they would like data on. The required metrics gather data on the average number of visitors per market day, total annual vendor sales at the market, average distance (in miles) traveled from product origin to the market (estimated annually), and acres in agricultural production by market vendors. These metrics are essential for the project because they serve as an important baseline for the other metrics. The analysis of these data shows market managers the impact they are having on a social, economic, and environmental level.
Table 1. Indicators for Impact Methods and Instruments
Methods:
Observation
Survey
Operations Research
Instruments:
-Visitor Count
-Market Events
-Vendor Sales Slips
-Visitor Survey
-Vendor Application
-Central Terminal Sales Receipts
- Volunteers
*This table represents the research methods adopted for this study, as well as the instruments associated with each method.
The other metrics are chosen based on the particular interests of each market. Examples of these metrics are: percentage of customers who were first time visitors; average number of SNAP transactions per year; number of different fruit and vegetable crops available for sale annually. This customizable feature allows for a wider range of markets to gather specific information, while still submitting data on the four metrics that are necessary for analyzing local, regional, and national impact.
The market managers are responsible for conducting and submitting completed surveys. Market managers have the option to print surveys and send them to the project team in Madison, or use an electronic method to capture results. The metrics are to be collected four times throughout the season. Because it is easier to see trends, the project team requested data be sampled on the third day of the market’s season; on a day where there is a market event, such as a live band or a speaker; on a non-event day in the middle of the season; and lastly, on the third day before the end of the season. Recording these four days allows the project team to interpret metrics better; for example, did attendance at the market increase due to an event occurring at the market, or did the market just see an overall increase of participation as it went further into the season.
These metrics, and their resulting data, allowed the project team to create long- and short-term analyses. Short-term analysis focused on the current economic, social and ecological impacts of farmers markets. Examples of measurable short-term effects include seeing how visitor count increased throughout the season, if events caused visitor count to increase, what percentage of market visitors that year used SNAP benefits, and how much money visitors planned on spending within and outside the market. Long-term analyses focused on what the markets could expect to see for future trends. Markets can use the long-term analysis to track the growth of the market, and create estimates regarding yearly spending in the area outside of the market. Overall, both data sets are going to allow markets to see their impact within the community. The trends can lend themselves as possible evidence for petitioning for government programs that support markets, enticing donors with the possibility of financial success, or strengthening community values and ties.
PARTICIPATE
Photo courtesy of Sassafras Creek Farms
Interested in joining Indicators for Impact?
Large or small; established or brand new; the project team is always looking for markets to participate
Indicators for Impact is helping markets learn about themselves for the first time
Following the 2015 market season, markets were able to upload their data into the web portal, while we began to clean and analyze the information. The data collected was useful for both the markets and research team. The portal aggregated the data into CSV (comma separated values) files, which is easily managed with both Excel and Google Sheets. Moreover, data cleaning helped the team address internal validity and fix errors related to entry, misinterpretations of questions, and overall use of the portal. In addition, the information from markets helped the research team understand how markets interpret the statements asked on the instruments. This is important because it brings us closer to using a common language that is both understandable by the research team and the market managers (Cotton, 2016).
The following analysis provides practical and tangible ways that market managers can use the DAIP instruments to develop short and long-term annotations:
Visitor Count
The visitor count instrument observed the average number of visitors per day. This data can be interpreted by using measures of central tendencies, such as finding a mean, median, and mode. The number of visits indicates the economic influence and community building capacity of the market. Furthermore, Market managers can use the long-term data to see which years brought in the most visitors, and if those years were correlated with significant market changes. Residents can use this data to measure markets impact on the local economy and use the information for community betterment, such as petitioning for increased funding for farmer markets.
Market Events
Documenting the number of market programs and events held (estimated annually) help managers plan the calendar of events at the beginning of the off season, such as festivals, cooking demonstrations, events for children and youth, music and more. Moreover, visitor surveys are distributed based on a sampling method. Surveys assess modes of transportation, zip codes, and the frequency rates amongst customers. Markets can use visitor counts to better inform their planning process and to highlight the many stakeholders that benefit from the market and its programs (Stephenson et al, 2011).
Vendor Sales Slips
Using the vendor sales slip, market managers can record weekly totals and analyze the market sale by type – farm products, prepared and processed food, ready-to-eat goods, and other specialty products. Not only can market managers use total sales to assess each vendor’s totals, but vendors too can utilize this multifunctional tool for recordkeeping, range (max and min) and forms of payments. Markets can utilize market sales to identify strengths and areas of success, as well as recommendations to increase accessibility to stakeholders (Lyson, Gillespie, and Hilchey, 1995). Finally, a record of payment types can help assist market managers with the developing and accessing infrastructure necessary to accommodate various forms of payment (Gerbasi 2006).
How did your market choose from the metrics offered through this project?
"At the time we selected our metrics, our future location was in question. We chose metrics that we thought would help us in grant writing and fundraising in our quest for a new permanent site. We are now much more secure with our current location. Some of our metrics were chosen as myself and many of our vendors are getting older [and so] we are interested in our future food supply, and the successful continuation of our market."
- Athens Farmers' Market
The Athens Farmers' Market exemplified how important it is for markets to see their economic impacts. Economic indicators can be selected for collection within multiple different surveys, including the Visitor Survey, the Vendor Sales slip, and the Vendor Application. The resulting data allows markets to see economic trends both within the market, and in the community outside the market. If the data from these metrics shows positive growth, they can be used as evidence for markets to ask for increased funding from donors or the government. The results can also highlight areas that the market might want to strengthen. As seen with the Athens market, managers can focus their metrics on gathering information about specific problems that markets may face in the future.
"To determine which metrics to collect, the board of directors and the market manager considered the information shared by the project leaders and then discussed what information would also be helpful in terms of long-term planning for our market. We also want to better understand who our customers are and where they come from."
-Chillicothe Farmers' Market
Measured with the Visitor Survey, the number of visits indicates the economic influence and community building capacity of the market. Market managers can use the long-term data to see which years brought in the most visitors, and if those years were correlated with significant market changes. Residents can use this data to measure market's impact on the local economy and use the information for community betterment, such as petitioning for increased funding for farmer markets. The Chillicothe Farmers' Market's focus on learning about their visitors lead them to choosing metrics on the Visitor Survey that collected data on how the visitors arrived to the market- car, bike, public transportation, etc.-, where they were coming from, and how often they would go to the market. Post-analysis, markets like Chillicothe might see that if a majority of their visitors are driving, it would be beneficial to petition for increased parking in the area.
"Crossroads wanted to expand, improve and streamline our data collection to include categories like vendor sales, product diversity and annual product donations. And, we're excited about having FMC/UW assistance collecting and interpreting data (that which we were already collecting) in a much more formal and organized way! This project is a huge help to Crossroads as we learn how and what to communicate externally. We've done some data collection in the past that was not very accurate and therefore not used. We chose metrics that will be of interest to our stakeholders and Board of Directors, but will not take too much time for our market team."
-Crossraods Farmers' Market
The Indicators for Impact survey is designed with time in mind; we know market managers are busy, so the surveys were created to be completed within a few minutes each hour or at the end of the market day. The versatile collection can be completed directly online if the market has access to internet; markets without internet access can use paper copies to collect data, and then electronically input the data in Excel or Google Sheets. The Vendor Sales slip measures the daily profit of the vendors and is administered after each market day. Daily collection of this information allows managers and vendors to see yearly growth trends and plan for the future.
"Being a part of the metric selection process allows us to communicate the health impacts of our market in the community. Our metric selection gives us the ability to show what fresh, local food are easily accessed at our market, how many people walk or bike to the market, and how the local economy is growing through farming, backyard farming and entrepreneurship."
-Hernando Farmers' Market
The Indicators for Impact program is the first of its kind in allowing markets to measure the impact they have on their communities, both within the market and outside of it. Because each market is able to customize their metric selection, markets are able to look at data that pertains specifically to their needs. This allows markets to make changes or predict future trends that will be beneficial to the community. The Hernando farmers' market was able to track the carbon footprint of their vendors and visitors, and used the data to see how having the farmers' market benefited the surrounding downtown area.
"Based on the positive experiences of markets with the study process, many anticipate continuing with the program again in 2016, and as a coordinating program we are hoping to expand the pool to include additional communities and/or weekday markets as well. Additionally, understanding the types of results that are possible through systematic survey instruments has increased interest by communities in better evaluating a variety of locally produced events, further increasing the effectiveness of organizations at communicating the value of downtown engagement to key audiences."
-Errin Wealty, Main Street Program
Errin Wealty is part of the Main Street Program, which was founded in 1980 to assist in the historic preservation of downtown areas found around Wisconsin. The Main Street Program has been working with the Indicators for Impact program for the past year gathering data on multiple markets around Wisconsin. The program's work with Errin has shown how entire states can work with the Indicators for Impact program to see state-wide trends in addition to single market trends. This larger data set is useful when applying for funding from a state-level, and also when tracking the carbon footprint of farmers and visitors across an entire state.
Limitations:
Throughout the pilot year, we saw six major challenges.
1. A lack of interest in starting the program due to difficult circumstances market managers are already experiencing. We worked around this limitation by recruiting nine markets from around the nation for the pilot study. The pilot markets experienced success in learning about their markets, and we expect this to generate interest in the program from other markets due to word of mouth.
2. Markets may have a difficult time choosing which metrics to use. We solve this problem by working directly with market managers. Because of the relationships we develop with the market managers, we are able to incorporate their values and goals into the metrics that they would want to select. The website features vignettes that can give market managers examples of how the project team has helped other markets achieve their personal goals.
3. Market managers have difficulty in incorporating the new data collection methods into the everyday running of the markets. Managers will be aware that we will work with them to help them best solve this critical problem. Through the training that the team will give the managers, and the personal support throughout the season, the managers should be comfortable adding the data collection process into their daily routines.
4. Managers might be interested in collecting more metrics than they are capable of. The team works to minimize this problem by working with managers to identify key metrics that managers can use to build interpretations. They also have the option to participate in the program for multiple years, allowing them to build a solid base in their first years before moving into more specific metric collection.
5. Market mortality, or markets not completing the data for the whole year. We seek to combat this crucial problem by staying in contact with market managers, and offering the support that they need to complete the season. Because it is a learning opportunity for both parties when markets drop out of the program, a brief evaluation would be conducted so that the project team can incorporate the market's feedback into future years to increase participation.
6. There might be more participation than the current project team can handle. If markets participate on a direct contract basis, we see no reason for us not being able to support the increased interest in the program. There is no limit on how many markets the online platform can hold. With a number of students and professionals interested in the work, labor force will not be a significant issue, and training videos relating to commonly seen problems can be recored and shared on platforms like YouTube.
Outside of the previously noted limitations, the project team noted other limitations that were consistent with the surveys wording and methods. Learning from these allows us to better prepare for the second year of study. There was an inconsistency with data collection. Not all markets uploaded data for each market day, which affects analysis related to annual sales and trends. In addition, some markets had missing or incomplete data. For example, in the vendor surveys, if an address was missing, the data was rendered invalid, because without verification, there was no way to maintain internal validity and credibility. Clarity errors were noted, due to potential language barriers, or misinterpretation of survey instructions. Some of the surveys had notes written on them relating to the unclear instructions or clarifying questions. Due to the variability of these notes, it was difficult to generate accurate descriptive statistics from them (Cotton, 2016).
Ultimately, despite the listed limitations, the Indicators for Impact program is important for markets to use to learn about themselves. Without the program, markets would have to rely on their own devices to come up with a data collection package that would not be comparable to any other market. With the Indicators for Impact, market managers receive direct support from the project team, detailed help with choosing the correct metrics for success, and are able to see data trend analyses that highlight the impact the markets have within their communities.
Pricing
Currently, the program has a price of $750 per market per year. This price includes access to the data collection instructions, data portal, analysis and info-graphics, and paragraphs supporting the market managers in interpreting their data. The project team is interested in working with both individual markets, and groups that manage multiple markets.
Contact Us
General Contact
Indicators for Impact Project
indicators4impact@wisc.edu
Administrative Staff
Project Leader- Alfonso Morales
(555)123-4567
indicators4impact@wisc.edu
Project Leader- Lauren Suerth
(555)123-4567
indicators4impact@wisc.edu
Summary and Conclusion
Based on the market feedback from the pilot year, the literature reviews, and the data we collected, we can confidently that market managers can benefit from participating in the Indicators for Impact program because it is the best way for them to collect data in a professional manner that gives them an overview of how they are impacting their communities. We recognize that this work is part of other great research projects taking flight, and that a necessary framework is being built for further assessment. Looking ahead to the final year of the AFRI project we have to opportunity to identify improvements and make necessary changes. A course website created by the undergraduate students working on this project may ultimately end up in the public domain and work in conjunction with the metric portal to provide market managers with the essential information needed to utilize this tool. Based on the initial success of the project, more markets (WI Main Street Market and Greenbay Market) will be using the most updated version of the DCP to collect data for this upcoming year (Cotton, 2016).
Dantrell Cotton took the lead in investigating how the Supplemental Nutrition Assistance Program is utilized in farmer's markets. His paper, Indicators For Impact, focuses on the impact SNAP has on annual market sales and food access barriers in under-served communities. The research expands and elaborates on the conventional hypothesis that when markets accept food assistances, sales increase. Though they express favorable sentiments in support of this hypothesis, they also offer a critical analysis on the complexities with establishing, maintaining and scaling-up nutrition incentives. The project team chose to focus on the SNAP usage at farmers markets because it has the opportunity to include many more people as visitors and it is has not been extensively studied before. Moreover, the project team recently learned that SNAP cards have an 8 digit code that can be tracked to see where people are using SNAP benefits around the nation.