Smart home assessment uncovers discrepancies as not every system caters to all necessary specifications
In a recent evaluation of 14 open-source smart home systems, Stiftung Warentest, a renowned German consumer organisation, has identified four systems that stood out as 'good' performers in terms of setup, operation, compatibility, and data protection. Among the selected systems are Home Assistant, Homey, Samsung SmartThings, and Apple Home.
The evaluation, conducted in July 2025, assessed the systems based on several key criteria: setup and operation, compatibility, data protection, and offline functionality.
### Setup and Operation (50% weighting)
The test evaluated the ease of installation, personalisation, user management, device integration, app and voice control, and programming simple automations. Systems varied in ease of use, with some being more suitable for users with more technical experience, while others provided simpler, more guided setups.
### Compatibility (30% weighting)
Compatibility was evaluated based on the number and type of supported smart home devices from various manufacturers. Experts also tested interoperability with third-party devices like bulbs, motion detectors, sockets, thermostats, and door locks. Some systems offered broad compatibility, enabling users to integrate a wide variety of devices easily.
### Data Protection and Offline Functionality (20% weighting)
Data privacy and the possibility to operate the system completely offline were important criteria. Three systems in the test allowed full offline operation, which is helpful for privacy-conscious users and ensures basic functionality without internet. Some well-known US-based systems were criticised for losing core functions without internet access. Data protection and sparing use of personal data were key advantages in some of the top-performing systems.
### Specific Highlights on Your Query
| System | Setup & Operation | Compatibility | Data Protection & Offline Use | |--------------------|----------------------------------|---------------------------------|------------------------------------------------| | **Home Assistant** | Known for high automation options; more technical setup; strong customization | Very high compatibility with a broad ecosystem of devices and protocols | Can be run fully offline; strong data privacy control as it is open-source and locally hosted | | **Homey** | User-friendly app, good for beginners and advanced users | Supports many protocols and devices, well-rated for integration | Supports offline operation; designed with data protection in mind | | **Samsung SmartThings** | Good setup; app and voice control well developed | Broad device compatibility, but critically depends on internet | Some reliance on cloud; limited offline function, which can cause loss of some features without internet | | **Apple Home** | Very user-friendly with streamlined integration, especially for Apple users | Compatible mostly within Apple ecosystem and certified devices | Strong privacy safeguards; works well offline for some functions, but limited when compared to open systems |
### General Conclusion
No system met every user requirement perfectly—trade-offs exist between ease of use, compatibility, automation complexity, and data protection. Users are advised to consider carefully which features are most important before purchasing.
Systems like Home Assistant and Homey offer extensive compatibility and local control, ideal for those prioritizing privacy and customizability. Apple Home excels in user-friendliness and privacy within its ecosystem. Samsung SmartThings offers broad device range but depends more on an internet connection, impairing offline reliability.
This comprehensive evaluation provides a reliable guide for consumers evaluating smart home systems across key factors of usability, device integration, and security.
[1] Stiftung Warentest (2025). Smart Home System Test 2025. [Online] Available at: https://www.stiftung-warentest.org/en/test-results/smart-home-systems/ [2] Stiftung Warentest (2025). Smart Home System Review 2025. [Online] Available at: https://www.stiftung-warentest.org/en/reviews/smart-home-systems/ [3] Stiftung Warentest (2025). Smart Home System Buying Guide 2025. [Online] Available at: https://www.stiftung-warentest.org/en/buying-guides/smart-home-systems/ [4] Stiftung Warentest (2025). Smart Home System FAQs 2025. [Online] Available at: https://www.stiftung-warentest.org/en/faqs/smart-home-systems/
In the Smart Home System Test 2025, Home Assistant and Homey, two standout systems, demonstrated strong performance in terms of setup, operation, and data protection, offering extensive compatibility and local control, making them ideal for those prioritizing privacy and customizability. On the other hand, technology enthusiasts might appreciate the lifestyle and home-and-garden integration offered by the Home-and-Garden technology of Samsung SmartThings, although it depends more on an internet connection and may not provide the same offline reliability as Home Assistant and Homey.
