Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

, # e.g., W001, FL12345\r\n 'Level': r'^Level\\s*\\d+$|^L\\d+$|^Уровень\\s*\\d+

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\r\n 'Email': r'^[\\w\\.-]+@[\\w\\.-]+\\.\\w+

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\r\n 'Phone': r'^\\+?\\d{10,15}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

\r\n }\r\n\r\n validity = {}\r\n for col, pattern in patterns.items():\r\n if col in self.df.columns:\r\n non_null = self.df[col].dropna()\r\n if len(non_null) > 0:\r\n matches = non_null.astype(str).str.match(pattern).sum()\r\n validity[col] = (matches / len(non_null) * 100)\r\n\r\n invalid = len(non_null) - matches\r\n if invalid > 0:\r\n self.issues.append(f\"{col}: {invalid} values don't match pattern\")\r\n else:\r\n validity[col] = 100\r\n\r\n overall = np.mean(list(validity.values())) if validity else 100\r\n\r\n self.results['validity'] = {\r\n 'by_column': validity,\r\n 'overall': overall,\r\n 'threshold': 95,\r\n 'passed': overall >= 95\r\n }\r\n\r\n return self.results['validity']\r\n\r\n def run_full_check(self):\r\n \"\"\"Run all quality checks\"\"\"\r\n self.check_completeness()\r\n self.check_accuracy()\r\n self.check_consistency()\r\n self.check_timeliness()\r\n self.check_validity()\r\n\r\n # Calculate overall score\r\n scores = []\r\n for metric in ['completeness', 'accuracy', 'consistency', 'validity']:\r\n if metric in self.results and self.results[metric].get('overall'):\r\n scores.append(self.results[metric]['overall'])\r\n\r\n self.results['overall_score'] = np.mean(scores) if scores else 0\r\n self.results['grade'] = self._calculate_grade(self.results['overall_score'])\r\n self.results['issues'] = self.issues\r\n\r\n return self.results\r\n\r\n def _calculate_grade(self, score):\r\n \"\"\"Calculate quality grade\"\"\"\r\n if score >= 98:\r\n return 'A+'\r\n elif score >= 95:\r\n return 'A'\r\n elif score >= 90:\r\n return 'B'\r\n elif score >= 80:\r\n return 'C'\r\n elif score >= 70:\r\n return 'D'\r\n else:\r\n return 'F'\r\n\r\n def generate_report(self):\r\n \"\"\"Generate quality report\"\"\"\r\n if not self.results:\r\n self.run_full_check()\r\n\r\n report = []\r\n report.append(\"=\" * 60)\r\n report.append(\"DATA QUALITY REPORT\")\r\n report.append(\"=\" * 60)\r\n report.append(f\"Records analyzed: {len(self.df)}\")\r\n report.append(f\"Columns: {len(self.df.columns)}\")\r\n report.append(\"\")\r\n report.append(f\"OVERALL SCORE: {self.results['overall_score']:.1f}% (Grade: {self.results['grade']})\")\r\n report.append(\"\")\r\n report.append(\"-\" * 60)\r\n\r\n # Detail by dimension\r\n for metric in ['completeness', 'accuracy', 'consistency', 'validity', 'timeliness']:\r\n if metric in self.results:\r\n r = self.results[metric]\r\n passed = '✓' if r.get('passed', False) else '✗'\r\n overall = r.get('overall', r.get('recent_percentage', 'N/A'))\r\n if isinstance(overall, (int, float)):\r\n report.append(f\"{metric.upper():15s}: {overall:>6.1f}% {passed}\")\r\n else:\r\n report.append(f\"{metric.upper():15s}: {overall}\")\r\n\r\n report.append(\"-\" * 60)\r\n\r\n if self.issues:\r\n report.append(\"\")\r\n report.append(\"ISSUES FOUND:\")\r\n for issue in self.issues[:10]: # Show first 10\r\n report.append(f\" • {issue}\")\r\n if len(self.issues) > 10:\r\n report.append(f\" ... and {len(self.issues) - 10} more issues\")\r\n\r\n report.append(\"\")\r\n report.append(\"=\" * 60)\r\n\r\n return \"\\n\".join(report)\r\n```\r\n\r\n## Validation Rules Builder\r\n\r\n### Custom Validation Rules\r\n\r\n```python\r\nclass ValidationRulesBuilder:\r\n \"\"\"Build custom validation rules for construction data\"\"\"\r\n\r\n def __init__(self):\r\n self.rules = []\r\n\r\n def add_not_null(self, column):\r\n \"\"\"Column must not have null values\"\"\"\r\n self.rules.append({\r\n 'type': 'not_null',\r\n 'column': column,\r\n 'check': lambda df, col=column: df[col].notna().all()\r\n })\r\n return self\r\n\r\n def add_unique(self, column):\r\n \"\"\"Column must have unique values\"\"\"\r\n self.rules.append({\r\n 'type': 'unique',\r\n 'column': column,\r\n 'check': lambda df, col=column: df[col].nunique() == len(df)\r\n })\r\n return self\r\n\r\n def add_range(self, column, min_val=None, max_val=None):\r\n \"\"\"Column values must be within range\"\"\"\r\n self.rules.append({\r\n 'type': 'range',\r\n 'column': column,\r\n 'min': min_val,\r\n 'max': max_val,\r\n 'check': lambda df, col=column, mn=min_val, mx=max_val:\r\n df[col].between(mn or -np.inf, mx or np.inf).all()\r\n })\r\n return self\r\n\r\n def add_regex(self, column, pattern):\r\n \"\"\"Column values must match regex pattern\"\"\"\r\n self.rules.append({\r\n 'type': 'regex',\r\n 'column': column,\r\n 'pattern': pattern,\r\n 'check': lambda df, col=column, p=pattern:\r\n df[col].astype(str).str.match(p).all()\r\n })\r\n return self\r\n\r\n def add_in_list(self, column, valid_values):\r\n \"\"\"Column values must be in list\"\"\"\r\n self.rules.append({\r\n 'type': 'in_list',\r\n 'column': column,\r\n 'valid_values': valid_values,\r\n 'check': lambda df, col=column, vals=valid_values:\r\n df[col].isin(vals).all()\r\n })\r\n return self\r\n\r\n def add_custom(self, name, check_func):\r\n \"\"\"Add custom validation function\"\"\"\r\n self.rules.append({\r\n 'type': 'custom',\r\n 'name': name,\r\n 'check': check_func\r\n })\r\n return self\r\n\r\n def validate(self, df):\r\n \"\"\"Run all validation rules\"\"\"\r\n results = []\r\n\r\n for rule in self.rules:\r\n try:\r\n passed = rule['check'](df)\r\n results.append({\r\n 'rule': rule.get('name', f\"{rule['type']}:{rule.get('column', 'custom')}\"),\r\n 'passed': passed,\r\n 'type': rule['type']\r\n })\r\n except Exception as e:\r\n results.append({\r\n 'rule': rule.get('name', f\"{rule['type']}:{rule.get('column', 'custom')}\"),\r\n 'passed': False,\r\n 'error': str(e)\r\n })\r\n\r\n return results\r\n\r\n# Usage example\r\nrules = (ValidationRulesBuilder()\r\n .add_not_null('ElementId')\r\n .add_unique('ElementId')\r\n .add_range('Volume_m3', min_val=0)\r\n .add_range('Cost', min_val=0)\r\n .add_in_list('Category', ['Wall', 'Floor', 'Column', 'Beam', 'Slab'])\r\n .add_regex('Level', r'^Level\\s*\\d+

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

)\r\n)\r\n\r\nresults = rules.validate(df)\r\nfor r in results:\r\n status = '✓' if r['passed'] else '✗'\r\n print(f\"{status} {r['rule']}\")\r\n```\r\n\r\n## Automated Quality Pipeline\r\n\r\n```python\r\nclass DataQualityPipeline:\r\n \"\"\"Automated data quality pipeline\"\"\"\r\n\r\n def __init__(self, config=None):\r\n self.config = config or self._default_config()\r\n self.history = []\r\n\r\n def _default_config(self):\r\n return {\r\n 'required_columns': ['ElementId', 'Category', 'Volume_m3'],\r\n 'unique_columns': ['ElementId'],\r\n 'numeric_ranges': {\r\n 'Volume_m3': (0, 10000),\r\n 'Area_m2': (0, 100000),\r\n 'Cost': (0, 100000000)\r\n },\r\n 'valid_categories': ['Wall', 'Floor', 'Column', 'Beam', 'Slab',\r\n 'Foundation', 'Roof', 'Stair', 'Door', 'Window'],\r\n 'min_quality_score': 90\r\n }\r\n\r\n def run(self, df, source_name='unknown'):\r\n \"\"\"Run quality pipeline\"\"\"\r\n checker = DataQualityChecker(df)\r\n\r\n # Configure checks based on config\r\n checker.check_completeness(self.config['required_columns'])\r\n checker.check_accuracy({\r\n col: {'min': r[0], 'max': r[1]}\r\n for col, r in self.config['numeric_ranges'].items()\r\n })\r\n checker.check_consistency(self.config['unique_columns'])\r\n checker.check_validity()\r\n\r\n results = checker.run_full_check()\r\n\r\n # Store in history\r\n self.history.append({\r\n 'timestamp': datetime.now(),\r\n 'source': source_name,\r\n 'records': len(df),\r\n 'score': results['overall_score'],\r\n 'grade': results['grade'],\r\n 'issues_count': len(results['issues'])\r\n })\r\n\r\n # Check threshold\r\n passed = results['overall_score'] >= self.config['min_quality_score']\r\n\r\n return {\r\n 'passed': passed,\r\n 'score': results['overall_score'],\r\n 'grade': results['grade'],\r\n 'details': results,\r\n 'report': checker.generate_report()\r\n }\r\n\r\n def get_history_summary(self):\r\n \"\"\"Get quality history summary\"\"\"\r\n if not self.history:\r\n return \"No quality checks performed yet.\"\r\n\r\n df_history = pd.DataFrame(self.history)\r\n return {\r\n 'total_checks': len(self.history),\r\n 'avg_score': df_history['score'].mean(),\r\n 'min_score': df_history['score'].min(),\r\n 'max_score': df_history['score'].max(),\r\n 'latest': self.history[-1]\r\n }\r\n```\r\n\r\n## Quality Reporting\r\n\r\n### Export Quality Report\r\n\r\n```python\r\ndef export_quality_report(df, output_path, include_details=True):\r\n \"\"\"Export comprehensive quality report to Excel\"\"\"\r\n checker = DataQualityChecker(df)\r\n results = checker.run_full_check()\r\n\r\n with pd.ExcelWriter(output_path, engine='openpyxl') as writer:\r\n # Summary sheet\r\n summary = pd.DataFrame({\r\n 'Metric': ['Overall Score', 'Grade', 'Records', 'Columns', 'Issues'],\r\n 'Value': [\r\n f\"{results['overall_score']:.1f}%\",\r\n results['grade'],\r\n len(df),\r\n len(df.columns),\r\n len(results['issues'])\r\n ]\r\n })\r\n summary.to_excel(writer, sheet_name='Summary', index=False)\r\n\r\n # Completeness details\r\n if 'completeness' in results:\r\n comp_df = pd.DataFrame.from_dict(\r\n results['completeness']['by_column'],\r\n orient='index',\r\n columns=['Completeness_%']\r\n )\r\n comp_df.to_excel(writer, sheet_name='Completeness')\r\n\r\n # Issues list\r\n if results['issues']:\r\n issues_df = pd.DataFrame({'Issue': results['issues']})\r\n issues_df.to_excel(writer, sheet_name='Issues', index=False)\r\n\r\n # Missing values analysis\r\n if include_details:\r\n missing = df.isnull().sum()\r\n missing_df = pd.DataFrame({\r\n 'Column': missing.index,\r\n 'Missing_Count': missing.values,\r\n 'Missing_%': (missing.values / len(df) * 100).round(2)\r\n })\r\n missing_df.to_excel(writer, sheet_name='Missing_Values', index=False)\r\n\r\n return output_path\r\n```\r\n\r\n## Quick Reference\r\n\r\n| Metric | Description | Threshold |\r\n|--------|-------------|-----------|\r\n| Completeness | % non-null values | ≥ 95% |\r\n| Accuracy | Values within valid range | ≥ 98% |\r\n| Consistency | Unique IDs, valid relationships | ≥ 99% |\r\n| Validity | Match expected patterns | ≥ 95% |\r\n| Timeliness | Records updated recently | ≥ 80% |\r\n\r\n## Common Validation Patterns\r\n\r\n```python\r\n# Construction-specific regex patterns\r\nPATTERNS = {\r\n 'element_id': r'^[A-Z]{1,3}\\d{3,8}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\r\n 'revit_id': r'^\\d{5,8}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\r\n 'ifc_guid': r'^[A-Za-z0-9_$]{22}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\r\n 'level': r'^(Level|L|Уровень)\\s*[-]?\\d+

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\r\n 'grid': r'^[A-Z]{1,2}[-/]?\\d{0,3}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\r\n 'date_iso': r'^\\d{4}-\\d{2}-\\d{2}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\r\n 'cost_code': r'^\\d{2,3}[.-]\\d{2,4}[.-]?\\d{0,4}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

\r\n}\r\n```\r\n\r\n## Resources\r\n\r\n- **Book**: \"Data-Driven Construction\" by Artem Boiko, Chapter 2.6\r\n- **Website**: https://datadrivenconstruction.io\r\n- **Great Expectations**: https://greatexpectations.io\r\n\r\n## Next Steps\r\n\r\n- See `bim-validation-pipeline` for BIM-specific validation\r\n- See `etl-pipeline` for data processing pipelines\r\n- See `data-visualization` for quality dashboards\r\n---","attachment_filenames":["_meta.json"],"attachments":[{"filename":"_meta.json","content":"{\n \"owner\": \"datadrivenconstruction\",\n \"slug\": \"data-quality-check\",\n \"displayName\": \"Data Quality Check\",\n \"latest\": {\n \"version\": \"2.1.0\",\n \"publishedAt\": 1771175403727,\n \"commit\": \"https://github.com/openclaw/skills/commit/1cda498632224950160205b224a63d1d0118b686\"\n },\n \"history\": [\n {\n \"version\": \"1.0.0\",\n \"publishedAt\": 1770475532210,\n \"commit\": \"https://github.com/openclaw/skills/commit/fc395dfc542d7afa685ff849c694d5d265e7cd8e\"\n }\n ]\n}\n","content_type":"application/json; charset=utf-8","language":"json","size":482,"content_sha256":"1c868ff36d2457fb11afab4199b42e7c2090e1197db8298e4e6a4ee722d03668"}],"content_json":{"type":"doc","content":[{"type":"heading","attrs":{"level":1},"content":[{"text":"Data Quality Check for Construction","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Overview","type":"text"}]},{"type":"paragraph","content":[{"text":"Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often.","type":"text"}]},{"type":"paragraph","content":[{"text":"Book Reference:","type":"text","marks":[{"type":"strong"}]},{"text":" \"Требования к качеству данных и его обеспечение\" / \"Data Quality Requirements\"","type":"text"}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"\"Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность.\" — DDC Book, Chapter 2.6","type":"text"}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Quick Start","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"python"},"content":[{"text":"import pandas as pd\n\n# Load construction data\ndf = pd.read_excel(\"bim_export.xlsx\")\n\n# Quick quality check\nquality_score = {\n 'completeness': (1 - df.isnull().sum().sum() / df.size) * 100,\n 'unique_ids': df['ElementId'].nunique() == len(df),\n 'valid_volumes': (df['Volume_m3'] >= 0).all()\n}\n\nprint(f\"Completeness: {quality_score['completeness']:.1f}%\")\nprint(f\"Unique IDs: {quality_score['unique_ids']}\")\nprint(f\"Valid volumes: {quality_score['valid_volumes']}\")","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Data Quality Dimensions","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"The 5 Quality Metrics","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"python"},"content":[{"text":"import pandas as pd\nimport numpy as np\nimport re\nfrom datetime import datetime, timedelta\n\nclass DataQualityChecker:\n \"\"\"Comprehensive data quality assessment for construction data\"\"\"\n\n def __init__(self, df):\n self.df = df.copy()\n self.results = {}\n self.issues = []\n\n def check_completeness(self, required_columns=None):\n \"\"\"Check for missing values (Полнота)\"\"\"\n if required_columns is None:\n required_columns = self.df.columns.tolist()\n\n completeness = {}\n for col in required_columns:\n if col in self.df.columns:\n non_null = self.df[col].notna().sum()\n total = len(self.df)\n completeness[col] = (non_null / total) * 100\n else:\n completeness[col] = 0\n self.issues.append(f\"Missing required column: {col}\")\n\n overall = np.mean(list(completeness.values()))\n\n self.results['completeness'] = {\n 'by_column': completeness,\n 'overall': overall,\n 'threshold': 95,\n 'passed': overall >= 95\n }\n\n return self.results['completeness']\n\n def check_accuracy(self, rules=None):\n \"\"\"Check data accuracy against rules (Точность)\"\"\"\n if rules is None:\n # Default construction data rules\n rules = {\n 'Volume_m3': {'min': 0, 'max': 10000},\n 'Area_m2': {'min': 0, 'max': 100000},\n 'Weight_kg': {'min': 0, 'max': 1000000},\n 'Cost': {'min': 0, 'max': 100000000}\n }\n\n accuracy = {}\n for col, bounds in rules.items():\n if col in self.df.columns:\n valid = self.df[col].between(\n bounds.get('min', -np.inf),\n bounds.get('max', np.inf)\n ).sum()\n total = self.df[col].notna().sum()\n accuracy[col] = (valid / total * 100) if total > 0 else 100\n\n # Log invalid values\n invalid_count = total - valid\n if invalid_count > 0:\n self.issues.append(\n f\"{col}: {invalid_count} values outside range [{bounds.get('min')}, {bounds.get('max')}]\"\n )\n\n overall = np.mean(list(accuracy.values())) if accuracy else 100\n\n self.results['accuracy'] = {\n 'by_column': accuracy,\n 'overall': overall,\n 'threshold': 98,\n 'passed': overall >= 98\n }\n\n return self.results['accuracy']\n\n def check_consistency(self, unique_cols=None, relationship_rules=None):\n \"\"\"Check data consistency (Согласованность)\"\"\"\n consistency = {}\n\n # Check unique columns\n if unique_cols is None:\n unique_cols = ['ElementId']\n\n for col in unique_cols:\n if col in self.df.columns:\n is_unique = self.df[col].nunique() == len(self.df)\n consistency[f'{col}_unique'] = 100 if is_unique else \\\n (self.df[col].nunique() / len(self.df) * 100)\n\n if not is_unique:\n duplicates = self.df[self.df[col].duplicated()][col].unique()\n self.issues.append(f\"Duplicate {col}: {len(duplicates)} duplicates found\")\n\n # Check cross-field relationships\n if relationship_rules is None:\n relationship_rules = [\n ('End_Date', '>=', 'Start_Date'),\n ('Gross_Volume', '>=', 'Net_Volume')\n ]\n\n for col1, op, col2 in relationship_rules:\n if col1 in self.df.columns and col2 in self.df.columns:\n if op == '>=':\n valid = (self.df[col1] >= self.df[col2]).sum()\n elif op == '>':\n valid = (self.df[col1] > self.df[col2]).sum()\n elif op == '==':\n valid = (self.df[col1] == self.df[col2]).sum()\n\n total = self.df[[col1, col2]].notna().all(axis=1).sum()\n consistency[f'{col1}_{op}_{col2}'] = (valid / total * 100) if total > 0 else 100\n\n overall = np.mean(list(consistency.values())) if consistency else 100\n\n self.results['consistency'] = {\n 'checks': consistency,\n 'overall': overall,\n 'threshold': 99,\n 'passed': overall >= 99\n }\n\n return self.results['consistency']\n\n def check_timeliness(self, date_col='Modified_Date', max_age_days=30):\n \"\"\"Check data timeliness (Своевременность)\"\"\"\n if date_col not in self.df.columns:\n self.results['timeliness'] = {\n 'overall': None,\n 'message': f'Column {date_col} not found'\n }\n return self.results['timeliness']\n\n dates = pd.to_datetime(self.df[date_col], errors='coerce')\n cutoff = datetime.now() - timedelta(days=max_age_days)\n\n recent = (dates >= cutoff).sum()\n total = dates.notna().sum()\n timeliness_pct = (recent / total * 100) if total > 0 else 0\n\n oldest = dates.min()\n newest = dates.max()\n avg_age = (datetime.now() - dates.mean()).days if dates.notna().any() else None\n\n self.results['timeliness'] = {\n 'recent_percentage': timeliness_pct,\n 'oldest_record': oldest,\n 'newest_record': newest,\n 'average_age_days': avg_age,\n 'threshold': 80,\n 'passed': timeliness_pct >= 80\n }\n\n return self.results['timeliness']\n\n def check_validity(self, patterns=None):\n \"\"\"Check data validity with regex patterns (Достоверность)\"\"\"\n if patterns is None:\n patterns = {\n 'ElementId': r'^[A-Z]{1,3}\\d{3,6}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

, # e.g., W001, FL12345\n 'Level': r'^Level\\s*\\d+$|^L\\d+$|^Уровень\\s*\\d+

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\n 'Email': r'^[\\w\\.-]+@[\\w\\.-]+\\.\\w+

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\n 'Phone': r'^\\+?\\d{10,15}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

\n }\n\n validity = {}\n for col, pattern in patterns.items():\n if col in self.df.columns:\n non_null = self.df[col].dropna()\n if len(non_null) > 0:\n matches = non_null.astype(str).str.match(pattern).sum()\n validity[col] = (matches / len(non_null) * 100)\n\n invalid = len(non_null) - matches\n if invalid > 0:\n self.issues.append(f\"{col}: {invalid} values don't match pattern\")\n else:\n validity[col] = 100\n\n overall = np.mean(list(validity.values())) if validity else 100\n\n self.results['validity'] = {\n 'by_column': validity,\n 'overall': overall,\n 'threshold': 95,\n 'passed': overall >= 95\n }\n\n return self.results['validity']\n\n def run_full_check(self):\n \"\"\"Run all quality checks\"\"\"\n self.check_completeness()\n self.check_accuracy()\n self.check_consistency()\n self.check_timeliness()\n self.check_validity()\n\n # Calculate overall score\n scores = []\n for metric in ['completeness', 'accuracy', 'consistency', 'validity']:\n if metric in self.results and self.results[metric].get('overall'):\n scores.append(self.results[metric]['overall'])\n\n self.results['overall_score'] = np.mean(scores) if scores else 0\n self.results['grade'] = self._calculate_grade(self.results['overall_score'])\n self.results['issues'] = self.issues\n\n return self.results\n\n def _calculate_grade(self, score):\n \"\"\"Calculate quality grade\"\"\"\n if score >= 98:\n return 'A+'\n elif score >= 95:\n return 'A'\n elif score >= 90:\n return 'B'\n elif score >= 80:\n return 'C'\n elif score >= 70:\n return 'D'\n else:\n return 'F'\n\n def generate_report(self):\n \"\"\"Generate quality report\"\"\"\n if not self.results:\n self.run_full_check()\n\n report = []\n report.append(\"=\" * 60)\n report.append(\"DATA QUALITY REPORT\")\n report.append(\"=\" * 60)\n report.append(f\"Records analyzed: {len(self.df)}\")\n report.append(f\"Columns: {len(self.df.columns)}\")\n report.append(\"\")\n report.append(f\"OVERALL SCORE: {self.results['overall_score']:.1f}% (Grade: {self.results['grade']})\")\n report.append(\"\")\n report.append(\"-\" * 60)\n\n # Detail by dimension\n for metric in ['completeness', 'accuracy', 'consistency', 'validity', 'timeliness']:\n if metric in self.results:\n r = self.results[metric]\n passed = '✓' if r.get('passed', False) else '✗'\n overall = r.get('overall', r.get('recent_percentage', 'N/A'))\n if isinstance(overall, (int, float)):\n report.append(f\"{metric.upper():15s}: {overall:>6.1f}% {passed}\")\n else:\n report.append(f\"{metric.upper():15s}: {overall}\")\n\n report.append(\"-\" * 60)\n\n if self.issues:\n report.append(\"\")\n report.append(\"ISSUES FOUND:\")\n for issue in self.issues[:10]: # Show first 10\n report.append(f\" • {issue}\")\n if len(self.issues) > 10:\n report.append(f\" ... and {len(self.issues) - 10} more issues\")\n\n report.append(\"\")\n report.append(\"=\" * 60)\n\n return \"\\n\".join(report)","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Validation Rules Builder","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Custom Validation Rules","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"python"},"content":[{"text":"class ValidationRulesBuilder:\n \"\"\"Build custom validation rules for construction data\"\"\"\n\n def __init__(self):\n self.rules = []\n\n def add_not_null(self, column):\n \"\"\"Column must not have null values\"\"\"\n self.rules.append({\n 'type': 'not_null',\n 'column': column,\n 'check': lambda df, col=column: df[col].notna().all()\n })\n return self\n\n def add_unique(self, column):\n \"\"\"Column must have unique values\"\"\"\n self.rules.append({\n 'type': 'unique',\n 'column': column,\n 'check': lambda df, col=column: df[col].nunique() == len(df)\n })\n return self\n\n def add_range(self, column, min_val=None, max_val=None):\n \"\"\"Column values must be within range\"\"\"\n self.rules.append({\n 'type': 'range',\n 'column': column,\n 'min': min_val,\n 'max': max_val,\n 'check': lambda df, col=column, mn=min_val, mx=max_val:\n df[col].between(mn or -np.inf, mx or np.inf).all()\n })\n return self\n\n def add_regex(self, column, pattern):\n \"\"\"Column values must match regex pattern\"\"\"\n self.rules.append({\n 'type': 'regex',\n 'column': column,\n 'pattern': pattern,\n 'check': lambda df, col=column, p=pattern:\n df[col].astype(str).str.match(p).all()\n })\n return self\n\n def add_in_list(self, column, valid_values):\n \"\"\"Column values must be in list\"\"\"\n self.rules.append({\n 'type': 'in_list',\n 'column': column,\n 'valid_values': valid_values,\n 'check': lambda df, col=column, vals=valid_values:\n df[col].isin(vals).all()\n })\n return self\n\n def add_custom(self, name, check_func):\n \"\"\"Add custom validation function\"\"\"\n self.rules.append({\n 'type': 'custom',\n 'name': name,\n 'check': check_func\n })\n return self\n\n def validate(self, df):\n \"\"\"Run all validation rules\"\"\"\n results = []\n\n for rule in self.rules:\n try:\n passed = rule['check'](df)\n results.append({\n 'rule': rule.get('name', f\"{rule['type']}:{rule.get('column', 'custom')}\"),\n 'passed': passed,\n 'type': rule['type']\n })\n except Exception as e:\n results.append({\n 'rule': rule.get('name', f\"{rule['type']}:{rule.get('column', 'custom')}\"),\n 'passed': False,\n 'error': str(e)\n })\n\n return results\n\n# Usage example\nrules = (ValidationRulesBuilder()\n .add_not_null('ElementId')\n .add_unique('ElementId')\n .add_range('Volume_m3', min_val=0)\n .add_range('Cost', min_val=0)\n .add_in_list('Category', ['Wall', 'Floor', 'Column', 'Beam', 'Slab'])\n .add_regex('Level', r'^Level\\s*\\d+

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

)\n)\n\nresults = rules.validate(df)\nfor r in results:\n status = '✓' if r['passed'] else '✗'\n print(f\"{status} {r['rule']}\")","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Automated Quality Pipeline","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"python"},"content":[{"text":"class DataQualityPipeline:\n \"\"\"Automated data quality pipeline\"\"\"\n\n def __init__(self, config=None):\n self.config = config or self._default_config()\n self.history = []\n\n def _default_config(self):\n return {\n 'required_columns': ['ElementId', 'Category', 'Volume_m3'],\n 'unique_columns': ['ElementId'],\n 'numeric_ranges': {\n 'Volume_m3': (0, 10000),\n 'Area_m2': (0, 100000),\n 'Cost': (0, 100000000)\n },\n 'valid_categories': ['Wall', 'Floor', 'Column', 'Beam', 'Slab',\n 'Foundation', 'Roof', 'Stair', 'Door', 'Window'],\n 'min_quality_score': 90\n }\n\n def run(self, df, source_name='unknown'):\n \"\"\"Run quality pipeline\"\"\"\n checker = DataQualityChecker(df)\n\n # Configure checks based on config\n checker.check_completeness(self.config['required_columns'])\n checker.check_accuracy({\n col: {'min': r[0], 'max': r[1]}\n for col, r in self.config['numeric_ranges'].items()\n })\n checker.check_consistency(self.config['unique_columns'])\n checker.check_validity()\n\n results = checker.run_full_check()\n\n # Store in history\n self.history.append({\n 'timestamp': datetime.now(),\n 'source': source_name,\n 'records': len(df),\n 'score': results['overall_score'],\n 'grade': results['grade'],\n 'issues_count': len(results['issues'])\n })\n\n # Check threshold\n passed = results['overall_score'] >= self.config['min_quality_score']\n\n return {\n 'passed': passed,\n 'score': results['overall_score'],\n 'grade': results['grade'],\n 'details': results,\n 'report': checker.generate_report()\n }\n\n def get_history_summary(self):\n \"\"\"Get quality history summary\"\"\"\n if not self.history:\n return \"No quality checks performed yet.\"\n\n df_history = pd.DataFrame(self.history)\n return {\n 'total_checks': len(self.history),\n 'avg_score': df_history['score'].mean(),\n 'min_score': df_history['score'].min(),\n 'max_score': df_history['score'].max(),\n 'latest': self.history[-1]\n }","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Quality Reporting","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Export Quality Report","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"python"},"content":[{"text":"def export_quality_report(df, output_path, include_details=True):\n \"\"\"Export comprehensive quality report to Excel\"\"\"\n checker = DataQualityChecker(df)\n results = checker.run_full_check()\n\n with pd.ExcelWriter(output_path, engine='openpyxl') as writer:\n # Summary sheet\n summary = pd.DataFrame({\n 'Metric': ['Overall Score', 'Grade', 'Records', 'Columns', 'Issues'],\n 'Value': [\n f\"{results['overall_score']:.1f}%\",\n results['grade'],\n len(df),\n len(df.columns),\n len(results['issues'])\n ]\n })\n summary.to_excel(writer, sheet_name='Summary', index=False)\n\n # Completeness details\n if 'completeness' in results:\n comp_df = pd.DataFrame.from_dict(\n results['completeness']['by_column'],\n orient='index',\n columns=['Completeness_%']\n )\n comp_df.to_excel(writer, sheet_name='Completeness')\n\n # Issues list\n if results['issues']:\n issues_df = pd.DataFrame({'Issue': results['issues']})\n issues_df.to_excel(writer, sheet_name='Issues', index=False)\n\n # Missing values analysis\n if include_details:\n missing = df.isnull().sum()\n missing_df = pd.DataFrame({\n 'Column': missing.index,\n 'Missing_Count': missing.values,\n 'Missing_%': (missing.values / len(df) * 100).round(2)\n })\n missing_df.to_excel(writer, sheet_name='Missing_Values', index=False)\n\n return output_path","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Quick Reference","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Metric","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Description","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Threshold","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Completeness","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"% non-null values","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"≥ 95%","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Accuracy","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Values within valid range","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"≥ 98%","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Consistency","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Unique IDs, valid relationships","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"≥ 99%","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Validity","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Match expected patterns","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"≥ 95%","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Timeliness","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Records updated recently","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"≥ 80%","type":"text"}]}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Common Validation Patterns","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"python"},"content":[{"text":"# Construction-specific regex patterns\nPATTERNS = {\n 'element_id': r'^[A-Z]{1,3}\\d{3,8}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\n 'revit_id': r'^\\d{5,8}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\n 'ifc_guid': r'^[A-Za-z0-9_$]{22}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\n 'level': r'^(Level|L|Уровень)\\s*[-]?\\d+

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\n 'grid': r'^[A-Z]{1,2}[-/]?\\d{0,3}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\n 'date_iso': r'^\\d{4}-\\d{2}-\\d{2}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

,\n 'cost_code': r'^\\d{2,3}[.-]\\d{2,4}[.-]?\\d{0,4}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…

\n}","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Resources","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Book","type":"text","marks":[{"type":"strong"}]},{"text":": \"Data-Driven Construction\" by Artem Boiko, Chapter 2.6","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Website","type":"text","marks":[{"type":"strong"}]},{"text":": https://datadrivenconstruction.io","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Great Expectations","type":"text","marks":[{"type":"strong"}]},{"text":": https://greatexpectations.io","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Next Steps","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"See ","type":"text"},{"text":"bim-validation-pipeline","type":"text","marks":[{"type":"code_inline"}]},{"text":" for BIM-specific validation","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"See ","type":"text"},{"text":"etl-pipeline","type":"text","marks":[{"type":"code_inline"}]},{"text":" for data processing pipelines","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"See ","type":"text"},{"text":"data-visualization","type":"text","marks":[{"type":"code_inline"}]},{"text":" for quality dashboards","type":"text"}]}]}]},{"type":"hr","attrs":{"markup":"---"}}]},"metadata":{"date":"2026-06-05","name":"data-quality-check","author":"@skillopedia","source":{"stars":2012,"repo_name":"openclaw-master-skills","origin_url":"https://github.com/leoyeai/openclaw-master-skills/blob/HEAD/skills/data-quality-check/SKILL.md","repo_owner":"leoyeai","body_sha256":"4ae9ebeda0333159f44aa57c7132137ceb4abb37797c24f884cc05ac2addb943","cluster_key":"65b9da79449836b05b6c9daefa031d4dba2922e364d05ac1363c503e50d56354","clean_bundle":{"format":"clean-skill-bundle-v1","source":"leoyeai/openclaw-master-skills/skills/data-quality-check/SKILL.md","attachments":[{"id":"9a74bfdc-a05a-5646-b7f5-4c2fca9395a0","key":"uploads/10433ee7-ad12-4ae0-b34e-97553e46c6c8/9a74bfdc-a05a-5646-b7f5-4c2fca9395a0/attachment.json","path":"_meta.json","size":482,"sha256":"1c868ff36d2457fb11afab4199b42e7c2090e1197db8298e4e6a4ee722d03668","contentType":"application/json; charset=utf-8"},{"id":"64c14dc6-625e-5916-9fa4-67781d572658","key":"uploads/10433ee7-ad12-4ae0-b34e-97553e46c6c8/64c14dc6-625e-5916-9fa4-67781d572658/attachment.json","path":"claw.json","size":595,"sha256":"692862f9b71b63073c65fe900d15af4ec3de389871ee0c7411bf74c82a57bea2","contentType":"application/json; charset=utf-8"},{"id":"deda26de-db95-50c9-b91f-d7964f52263a","key":"uploads/10433ee7-ad12-4ae0-b34e-97553e46c6c8/deda26de-db95-50c9-b91f-d7964f52263a/attachment.md","path":"instructions.md","size":1284,"sha256":"bd05074ff38e1c0d3e186fb80b4df604228193883c9d0858fb23d6940c7dcdda","contentType":"text/markdown; charset=utf-8"}],"bundle_sha256":"ceaad8babdf8f31e3267d7b3f955f15719c7d9ea4d25073ec09dd6f26f11ddd9","attachment_count":3,"text_attachments":3,"attachment_storage":"skillopedia-attachments-v1","binary_attachments":0,"excluded_attachments":[]},"cluster_size":2,"skill_md_path":"skills/data-quality-check/SKILL.md","import_metadata":{"date":"2026-06-05","author":"@skillopedia","version":"v1","category":"data-analytics","category_label":"Data"},"exact_dupes_collapsed_into_this":1},"version":"v1","category":"data-analytics","homepage":"https://datadrivenconstruction.io","metadata":{"openclaw":{"os":["win32"],"emoji":"✔️","homepage":"https://datadrivenconstruction.io","requires":{"bins":["python3"]}}},"import_tag":"clean-skills-v1","description":"Assess construction data quality using completeness, accuracy, consistency, timeliness, and validity metrics. Automated validation with regex patterns, thresholds, and reporting."}},"renderedAt":1782980136051}

Data Quality Check for Construction Overview Based on DDC methodology (Chapter 2.6), this skill provides comprehensive data quality assessment for construction projects. Poor data quality leads to poor decisions - validate early, validate often. Book Reference: "Требования к качеству данных и его обеспечение" / "Data Quality Requirements" "Качество данных определяется пятью ключевыми метриками: полнота, точность, согласованность, своевременность и достоверность." — DDC Book, Chapter 2.6 Quick Start Data Quality Dimensions The 5 Quality Metrics Validation Rules Builder Custom Validation Rules…