Research Methodology
Last updated: July 2026
Quick Answer
PilotStack produces original software market research using primary surveys, hands-on testing data, and secondary sources including G2, Capterra, and analyst reports. Every research project follows a 6-stage process: question definition, data collection, analysis, peer review, publication, and updates. A minimum 20% of all data points are independently validated.
TL;DR
- Research draws from primary surveys, hands-on testing, and secondary sources (G2, Capterra, analyst reports)
- 6-stage process: question definition, data collection, analysis, peer review, publication, updates
- Minimum 20% of data points independently validated by second researcher
- Pricing data verified against vendor websites at time of collection
- All limitations and methodology notes published transparently with each report
Key Takeaways
- Each research project starts with a specific question or hypothesis before data collection begins
- Data collected systematically with documented collection dates and methods for every data point
- Pricing data includes multiple tiers and regions where available
- Outliers investigated and documented; 20% sample independently validated
- Peer review by researcher not involved in original collection
- Detailed methodology, sample sizes, and limitations published with every report
- Research updated annually or when significant market changes occur
- Data visualizations include source attribution, collection dates, and accessible text alternatives
Beyond individual product reviews, PilotStack produces original research reports, market analyses, pricing benchmarks, and industry trend reports. This page documents how we conduct this research.
Data Collection
Our research draws from multiple data sources to ensure comprehensive and accurate findings:
- Primary research: Original surveys of software buyers and vendors, pricing analysis from public sources and direct vendor quotes, and feature comparison data from our hands-on testing process.
- Secondary research: Aggregated and anonymized data from G2, Capterra, and TrustRadius reviews; publicly available analyst reports from Gartner, Forrester, and IDC; vendor-published case studies, whitepapers, and documentation; industry benchmarks from published sources.
- Market data: Pricing information collected from vendor websites, public pricing pages, and direct verification with sales teams. We collect pricing for multiple tiers and regions where available.
Research Process
1.Research Question Definition
Each research project begins with a specific research question or hypothesis aligned with our editorial mission. We define the scope, methodology, and data sources before data collection begins.
2.Data Collection
Data is collected systematically according to the project-specific methodology. For pricing research, we collect data from vendor websites, verify with sales teams where possible, and document the collection date and method for every data point.
3.Analysis & Validation
Data is analyzed using appropriate statistical methods. A second researcher independently validates a minimum 20% sample of the data to ensure accuracy. Outliers are investigated and documented.
4.Peer Review
Every research report undergoes internal peer review by at least one researcher not involved in the original data collection. The reviewer evaluates methodology, data accuracy, and interpretation.
5.Publication & Transparency
Published research includes detailed methodology notes, data collection dates, sample sizes, and limitations. We clearly distinguish between findings supported by our data and interpretive analysis.
6.Updates
Research reports include publication dates and are updated annually or when significant market changes warrant earlier revision. Updates are noted with the date and nature of changes.
Data Quality Standards
- All data points include source attribution and collection date
- Pricing data is verified against vendor websites at the time of collection
- Survey data includes documented sample sizes, methodologies, and margin of error where applicable
- Secondary data sources are evaluated for recency, methodology quality, and potential bias
- Data visualizations accurately represent underlying data with clear axis labels and source notes
Original Charts and Visualizations
Charts and data visualizations in our research reports are created by our team using verified data. Each chart includes:
- Clear title and axis labels
- Data source attribution with collection date
- Methodology notes for complex visualizations
- Accessible text alternatives for screen readers
Limitations
We acknowledge the following limitations in our research:
- Pricing data reflects publicly available information and may not include enterprise discounts, negotiated pricing, or region-specific variations
- User review data is aggregated from third-party platforms and reflects the opinions of their user bases, which may not be representative of all users
- Market size and growth estimates are based on published analyst reports and may use different methodologies and definitions
- Our research focuses primarily on English-language tools and North American/European markets
For questions about our research methodology, contact us at [email protected].