Jul 28 17:10:15.943988 ip-10-0-139-193 systemd[1]: Starting Kubernetes Kubelet... Jul 28 17:10:16.380151 ip-10-0-139-193 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:10:16.380151 ip-10-0-139-193 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 28 17:10:16.380151 ip-10-0-139-193 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:10:16.380151 ip-10-0-139-193 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 28 17:10:16.380151 ip-10-0-139-193 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:10:16.380854 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.380177 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 28 17:10:16.385461 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385444 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:10:16.385461 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385460 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385464 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385467 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385470 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385473 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385476 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385479 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385482 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385484 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385487 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385490 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385494 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385496 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385499 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385502 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385505 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385508 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385510 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385514 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385516 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:10:16.385533 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385523 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385526 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385529 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385537 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385540 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385543 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385546 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385549 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385551 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385554 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385557 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385560 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385562 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385565 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385567 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385571 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385575 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385578 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385580 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:10:16.386036 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385583 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385585 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385588 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385590 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385593 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385595 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385597 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385600 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385602 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385605 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385607 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385610 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385612 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385615 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385618 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385621 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385624 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385627 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385631 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385635 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:10:16.386534 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385638 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385641 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385644 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385647 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385649 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385652 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385656 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385659 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385662 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385665 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385668 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385671 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385674 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385677 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385680 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385682 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385685 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385688 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385690 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385693 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:10:16.387055 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385696 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385698 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385700 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385703 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385724 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385727 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385730 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385734 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385736 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385739 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385742 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385745 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385747 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385750 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385753 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385756 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385759 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385762 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.385764 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386447 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:10:16.387870 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386453 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386456 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386458 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386462 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386465 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386469 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386472 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386474 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386477 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386480 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386482 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386485 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386487 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386490 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386493 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386495 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386498 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386501 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386503 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386506 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:10:16.388402 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386509 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386512 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386517 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386521 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386524 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386527 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386530 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386533 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386536 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386539 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386542 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386545 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386547 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386550 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386552 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386556 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386559 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386562 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386565 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:10:16.389041 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386568 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386571 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386574 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386576 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386579 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386581 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386584 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386586 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386589 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386591 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386594 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386596 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386600 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386604 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386608 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386611 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386614 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386616 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386619 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386621 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:10:16.389546 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386624 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386628 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386630 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386633 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386636 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386638 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386641 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386644 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386647 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386649 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386652 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386654 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386657 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386659 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386662 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386664 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386667 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386669 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386672 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386675 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:10:16.390107 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386677 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386679 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386682 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386685 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386688 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386691 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386693 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386697 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386699 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386702 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386718 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386721 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386724 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386727 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386729 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386733 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386735 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386738 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.386740 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390317 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:10:16.390637 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390331 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390335 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390346 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390349 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390370 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390484 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390490 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390493 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390496 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390499 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390502 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390505 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390507 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390510 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390513 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390517 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390521 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390525 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390527 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390530 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:10:16.391159 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390533 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390536 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390539 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390545 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390548 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390550 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390553 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390556 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390559 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390561 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390564 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390567 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390570 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390573 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390576 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390579 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390581 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390586 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390589 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390591 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:10:16.391643 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390594 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390596 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390599 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390603 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390608 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390611 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390615 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390618 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390620 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390624 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390627 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390630 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390632 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390635 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390637 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390640 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390643 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390647 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390650 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390654 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:10:16.392181 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390657 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390661 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390664 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390668 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390672 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390675 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390678 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390681 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390684 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390687 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390689 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390692 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390694 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390697 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390700 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390702 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390717 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390720 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390723 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:10:16.392668 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390725 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390728 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390731 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390734 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390737 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390739 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390742 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390744 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390747 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390750 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390752 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390755 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390758 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390761 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390763 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390766 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390769 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390772 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.390775 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390844 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390851 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 28 17:10:16.393139 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390857 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390861 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390866 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390869 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390873 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390877 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390880 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390883 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390887 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390890 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390893 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390896 2562 flags.go:64] FLAG: --cgroup-root="" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390899 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390902 2562 flags.go:64] FLAG: --client-ca-file="" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390904 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390908 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390913 2562 flags.go:64] FLAG: --cluster-domain="" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390915 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390918 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390921 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390925 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390929 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390932 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 28 17:10:16.393672 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390935 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390939 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390942 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390945 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390948 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390951 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390954 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390959 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390961 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390964 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390976 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390980 2562 flags.go:64] FLAG: --enable-server="true" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390983 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390988 2562 flags.go:64] FLAG: --event-burst="100" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390991 2562 flags.go:64] FLAG: --event-qps="50" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390994 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.390997 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391000 2562 flags.go:64] FLAG: --eviction-hard="" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391004 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391007 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391010 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391013 2562 flags.go:64] FLAG: --eviction-soft="" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391016 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391019 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391022 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 28 17:10:16.394244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391025 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391028 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391031 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391034 2562 flags.go:64] FLAG: --feature-gates="" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391053 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391058 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391061 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391064 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391067 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391070 2562 flags.go:64] FLAG: --help="false" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391073 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391077 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391081 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391084 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391087 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391091 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391094 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391097 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391099 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391102 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391105 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391108 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391111 2562 flags.go:64] FLAG: --kube-reserved="" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391114 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 28 17:10:16.394897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391117 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391120 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391123 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391125 2562 flags.go:64] FLAG: --lock-file="" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391128 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391132 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391135 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391141 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391144 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391147 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391150 2562 flags.go:64] FLAG: --logging-format="text" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391153 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391156 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391160 2562 flags.go:64] FLAG: --manifest-url="" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391163 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391168 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391171 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391176 2562 flags.go:64] FLAG: --max-pods="110" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391178 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391182 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391184 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391188 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391190 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391194 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391197 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 28 17:10:16.395500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391211 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391214 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391217 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391221 2562 flags.go:64] FLAG: --pod-cidr="" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391223 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391229 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391232 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391235 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391238 2562 flags.go:64] FLAG: --port="10250" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391241 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391244 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-059b66c7421a91adc" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391247 2562 flags.go:64] FLAG: --qos-reserved="" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391250 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391255 2562 flags.go:64] FLAG: --register-node="true" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391258 2562 flags.go:64] FLAG: --register-with-taints="" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391264 2562 flags.go:64] FLAG: --registry-burst="10" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391267 2562 flags.go:64] FLAG: --registry-qps="5" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391270 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391273 2562 flags.go:64] FLAG: --reserved-memory="" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391277 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391280 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391283 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391286 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391288 2562 flags.go:64] FLAG: --runonce="false" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391291 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 28 17:10:16.396121 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391294 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391297 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391300 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391303 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391306 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391309 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391312 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391315 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391318 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391321 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391324 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391327 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391330 2562 flags.go:64] FLAG: --system-cgroups="" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391333 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391339 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391342 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391345 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391348 2562 flags.go:64] FLAG: --tls-min-version="" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391351 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391354 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391359 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391362 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391365 2562 flags.go:64] FLAG: --v="2" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391369 2562 flags.go:64] FLAG: --version="false" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391373 2562 flags.go:64] FLAG: --vmodule="" Jul 28 17:10:16.396747 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391378 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.391381 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391491 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391495 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391498 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391501 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391504 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391507 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391510 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391513 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391515 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391518 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391520 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391523 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391526 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391528 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391531 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391534 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391536 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391539 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391541 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:10:16.397388 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391544 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391546 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391549 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391551 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391554 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391556 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391560 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391563 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391566 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391568 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391571 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391573 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391576 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391579 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391582 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391584 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391587 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391589 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391592 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391594 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:10:16.397938 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391597 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391599 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391602 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391605 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391607 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391611 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391616 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391619 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391622 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391625 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391627 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391630 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391633 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391636 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391639 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391641 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391644 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391646 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391651 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:10:16.398424 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391656 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391660 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391663 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391666 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391669 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391671 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391675 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391677 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391681 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391683 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391686 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391688 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391691 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391693 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391696 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391698 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391701 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391704 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391718 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:10:16.398925 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391722 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391725 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391727 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391730 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391733 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391735 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391738 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391740 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391743 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391746 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391749 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391751 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391755 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391759 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391762 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391765 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391767 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391770 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391773 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:10:16.399430 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391775 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:10:16.399910 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391778 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:10:16.399910 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.391781 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:10:16.399910 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.392578 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:10:16.400684 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.400664 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 28 17:10:16.400730 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.400685 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 28 17:10:16.400730 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.400698 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 28 17:10:16.400730 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.400720 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 28 17:10:16.400814 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400776 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:10:16.400814 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400780 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:10:16.400814 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400784 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:10:16.400814 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400788 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:10:16.400814 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400791 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:10:16.400814 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400794 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:10:16.400814 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400797 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:10:16.400814 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400800 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:10:16.400814 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400804 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:10:16.400814 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400806 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:10:16.400814 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400809 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:10:16.400814 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400812 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:10:16.400814 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400815 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:10:16.400814 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400818 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:10:16.400814 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400821 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400824 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400826 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400829 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400831 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400834 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400837 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400839 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400842 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400845 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400847 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400850 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400852 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400855 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400857 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400861 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400864 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400866 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400869 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400871 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:10:16.401176 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400874 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400877 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400879 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400882 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400884 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400887 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400890 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400893 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400895 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400898 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400902 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400905 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400909 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400911 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400915 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400918 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400921 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400924 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:10:16.401659 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400928 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400931 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400934 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400936 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400939 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400942 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400945 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400948 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400951 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400954 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400958 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400960 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400963 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400965 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400968 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400970 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400973 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400975 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400978 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:10:16.402174 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400980 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400983 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400985 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400988 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400990 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400992 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400995 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.400997 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401000 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401003 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401006 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401008 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401011 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401014 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401016 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401019 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401022 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401025 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401027 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401029 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:10:16.402652 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401032 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:10:16.403154 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401035 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:10:16.403154 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401038 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:10:16.403154 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401040 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:10:16.403154 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401043 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:10:16.403154 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401046 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:10:16.403154 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401049 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:10:16.403154 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401051 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:10:16.403154 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.401056 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:10:16.403154 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401166 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:10:16.403154 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401170 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:10:16.403154 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401173 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:10:16.403154 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401176 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:10:16.403154 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401179 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:10:16.403154 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401181 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:10:16.403154 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401184 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401187 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401190 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401193 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401195 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401198 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401201 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401203 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401206 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401208 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401211 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401213 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401216 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401218 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401221 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401223 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401226 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401229 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401231 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:10:16.403515 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401234 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401236 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401239 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401243 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401246 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401249 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401251 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401254 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401257 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401261 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401264 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401267 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401269 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401272 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401275 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401277 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401280 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401283 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401285 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401288 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:10:16.403987 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401290 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401293 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401296 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401298 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401301 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401303 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401306 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401308 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401311 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401313 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401316 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401318 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401320 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401323 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401325 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401328 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401331 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401333 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401336 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401338 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:10:16.404473 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401341 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401343 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401346 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401348 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401351 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401353 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401356 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401359 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401361 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401363 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401367 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401369 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401373 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401376 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401379 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401382 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401385 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401388 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401391 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:10:16.404969 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401394 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:10:16.405455 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401397 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:10:16.405455 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401399 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:10:16.405455 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401402 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:10:16.405455 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401405 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:10:16.405455 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401407 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:10:16.405455 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401410 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:10:16.405455 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401413 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:10:16.405455 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401415 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:10:16.405455 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401418 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:10:16.405455 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401422 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:10:16.405455 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401424 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:10:16.405455 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401427 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:10:16.405455 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401430 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:10:16.405455 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:16.401433 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:10:16.405455 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.401438 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:10:16.405836 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.401544 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 28 17:10:16.405836 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.405072 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 28 17:10:16.406597 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.406585 2562 server.go:1019] "Starting client certificate rotation" Jul 28 17:10:16.406727 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.406696 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 17:10:16.406792 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.406760 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 17:10:16.409341 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.409325 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 17:10:16.414387 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.414368 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 17:10:16.423013 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.422994 2562 log.go:25] "Validated CRI v1 runtime API" Jul 28 17:10:16.425114 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.425101 2562 log.go:25] "Validated CRI v1 image API" Jul 28 17:10:16.426645 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.426625 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 28 17:10:16.430374 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.430354 2562 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c9f0c1e2-919f-43f2-9834-f8ebf0b307af:/dev/nvme0n1p4 fa907065-e093-432e-b33f-811e1b9e48d8:/dev/nvme0n1p3] Jul 28 17:10:16.430438 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.430372 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 28 17:10:16.437511 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.437490 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 17:10:16.437618 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.437464 2562 manager.go:216] Machine: {Timestamp:2026-07-28 17:10:16.435887036 +0000 UTC m=+0.367506385 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3127888 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2688a899d616692d14551c626cdb69 SystemUUID:ec2688a8-99d6-1669-2d14-551c626cdb69 BootID:04df451b-2202-46b1-8340-3234123f1eb2 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e7:df:6c:90:79 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e7:df:6c:90:79 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:67:4f:a1:fe:05 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 28 17:10:16.437689 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.437622 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 28 17:10:16.437756 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.437743 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 28 17:10:16.441416 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.441393 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 28 17:10:16.441553 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.441419 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-193.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 28 17:10:16.442206 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.442196 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 28 17:10:16.442244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.442209 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 28 17:10:16.442244 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.442216 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 17:10:16.443016 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.443005 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 17:10:16.443858 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.443849 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 28 17:10:16.445614 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.445603 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 28 17:10:16.445742 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.445733 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 28 17:10:16.445785 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.445763 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 28 17:10:16.445785 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.445773 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 28 17:10:16.445902 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.445787 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 28 17:10:16.445902 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.445797 2562 kubelet.go:398] "Adding apiserver pod source" Jul 28 17:10:16.445902 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.445806 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 28 17:10:16.446153 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.446141 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 17:10:16.446201 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.446161 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 17:10:16.448533 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.448516 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 28 17:10:16.450542 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.450529 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 28 17:10:16.450588 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.450560 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 28 17:10:16.452031 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.452019 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 28 17:10:16.452079 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.452034 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 28 17:10:16.452079 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.452040 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 28 17:10:16.452079 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.452045 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 28 17:10:16.452079 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.452055 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 28 17:10:16.452079 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.452063 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 28 17:10:16.452079 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.452072 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 28 17:10:16.452079 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.452078 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 28 17:10:16.452248 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.452085 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 28 17:10:16.452248 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.452091 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 28 17:10:16.452248 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.452104 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 28 17:10:16.452248 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.452110 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 28 17:10:16.454197 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.454183 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 28 17:10:16.454197 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.454196 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 28 17:10:16.455793 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.455775 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-193.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 28 17:10:16.455857 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.455821 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 28 17:10:16.455857 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.455825 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-193.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 28 17:10:16.457233 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.457220 2562 server.go:1268] "Started kubelet" Jul 28 17:10:16.457328 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.457304 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 28 17:10:16.457386 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.457300 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 28 17:10:16.457386 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.457369 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 28 17:10:16.458115 ip-10-0-139-193 systemd[1]: Started Kubernetes Kubelet. Jul 28 17:10:16.460124 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.460100 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 28 17:10:16.460464 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.460446 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 28 17:10:16.463895 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.463876 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 28 17:10:16.463969 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.462768 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-193.ec2.internal.18c68340c222e2cb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-193.ec2.internal,UID:ip-10-0-139-193.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-193.ec2.internal,},FirstTimestamp:2026-07-28 17:10:16.457192139 +0000 UTC m=+0.388811488,LastTimestamp:2026-07-28 17:10:16.457192139 +0000 UTC m=+0.388811488,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-193.ec2.internal,}" Jul 28 17:10:16.467269 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.467251 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 28 17:10:16.467269 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.467260 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 28 17:10:16.467450 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.467429 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:16.467940 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.467915 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 28 17:10:16.468048 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.467920 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 28 17:10:16.468048 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.467964 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 28 17:10:16.468347 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.468329 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 28 17:10:16.468347 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.468347 2562 factory.go:55] Registering systemd factory Jul 28 17:10:16.468478 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.468355 2562 factory.go:223] Registration of the systemd container factory successfully Jul 28 17:10:16.468587 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.468575 2562 factory.go:153] Registering CRI-O factory Jul 28 17:10:16.468636 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.468589 2562 factory.go:223] Registration of the crio container factory successfully Jul 28 17:10:16.468636 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.468613 2562 factory.go:103] Registering Raw factory Jul 28 17:10:16.468688 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.468678 2562 manager.go:1196] Started watching for new ooms in manager Jul 28 17:10:16.468969 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.468944 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 28 17:10:16.469076 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.469055 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 28 17:10:16.469248 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.469236 2562 manager.go:349] Starting recovery of all containers Jul 28 17:10:16.470050 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.470025 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-26bd9" Jul 28 17:10:16.479637 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.479608 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-26bd9" Jul 28 17:10:16.480698 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.480676 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 28 17:10:16.480698 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.480654 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-193.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 28 17:10:16.483528 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.483512 2562 manager.go:354] Recovery completed Jul 28 17:10:16.485287 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.485059 2562 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 28 17:10:16.488428 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.488416 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:10:16.490194 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.490175 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:10:16.490279 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.490202 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:10:16.490279 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.490211 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:10:16.490817 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.490800 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 28 17:10:16.490911 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.490815 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:10:16.490911 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.490817 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 28 17:10:16.490911 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.490865 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 28 17:10:16.491494 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.491478 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:10:16.491568 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.491507 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:10:16.491568 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.491520 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:10:16.492699 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.492686 2562 policy_none.go:49] "None policy: Start" Jul 28 17:10:16.492765 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.492718 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 28 17:10:16.492765 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.492732 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 28 17:10:16.493919 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.493905 2562 policy_none.go:47] "Start" Jul 28 17:10:16.521298 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.521281 2562 manager.go:337] "Starting Device Plugin manager" Jul 28 17:10:16.521398 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.521318 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 28 17:10:16.521398 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.521331 2562 server.go:87] "Starting device plugin registration server" Jul 28 17:10:16.521601 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.521587 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 28 17:10:16.521656 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.521603 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 28 17:10:16.521885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.521870 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 28 17:10:16.521967 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.521954 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 28 17:10:16.522016 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.521968 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 28 17:10:16.522348 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.522324 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 28 17:10:16.522424 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.522387 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:16.540335 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.540314 2562 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2596 into the system container \"/system.slice\": " Jul 28 17:10:16.607499 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.607457 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 28 17:10:16.608691 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.608671 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 28 17:10:16.608798 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.608730 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 28 17:10:16.608798 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.608762 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 28 17:10:16.608909 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.608804 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 28 17:10:16.610995 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.610974 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:10:16.621716 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.621679 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:10:16.622514 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.622499 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:10:16.622585 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.622530 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:10:16.622585 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.622540 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:10:16.622585 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.622564 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.635132 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.635087 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.635132 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.635109 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-193.ec2.internal\": node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:16.660392 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.660372 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:16.709477 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.709453 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-193.ec2.internal"] Jul 28 17:10:16.709568 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.709538 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:10:16.710916 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.710900 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:10:16.711013 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.710936 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:10:16.711013 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.710951 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:10:16.712227 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.712212 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:10:16.712336 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.712321 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.712396 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.712353 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:10:16.712925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.712903 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:10:16.712925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.712916 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:10:16.713054 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.712933 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:10:16.713054 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.712946 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:10:16.713127 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.712935 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:10:16.713127 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.713076 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:10:16.714135 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.714122 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.714189 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.714153 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:10:16.714805 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.714789 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:10:16.714868 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.714820 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:10:16.714868 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.714835 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:10:16.739240 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.739217 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-193.ec2.internal\" not found" node="ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.743536 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.743522 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-193.ec2.internal\" not found" node="ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.761263 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.761244 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:16.771184 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.771157 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a24aeb13dfbd469c43b0e4dc033b65ed-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal\" (UID: \"a24aeb13dfbd469c43b0e4dc033b65ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.771276 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.771196 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a24aeb13dfbd469c43b0e4dc033b65ed-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal\" (UID: \"a24aeb13dfbd469c43b0e4dc033b65ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.771276 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.771223 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0e753e1f8e3d27215843b4f99d76207a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-193.ec2.internal\" (UID: \"0e753e1f8e3d27215843b4f99d76207a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.861970 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.861939 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:16.872115 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.872096 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a24aeb13dfbd469c43b0e4dc033b65ed-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal\" (UID: \"a24aeb13dfbd469c43b0e4dc033b65ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.872204 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.872124 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a24aeb13dfbd469c43b0e4dc033b65ed-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal\" (UID: \"a24aeb13dfbd469c43b0e4dc033b65ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.872204 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.872141 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0e753e1f8e3d27215843b4f99d76207a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-193.ec2.internal\" (UID: \"0e753e1f8e3d27215843b4f99d76207a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.872286 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.872201 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0e753e1f8e3d27215843b4f99d76207a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-193.ec2.internal\" (UID: \"0e753e1f8e3d27215843b4f99d76207a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.872286 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.872207 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a24aeb13dfbd469c43b0e4dc033b65ed-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal\" (UID: \"a24aeb13dfbd469c43b0e4dc033b65ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.872286 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:16.872238 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a24aeb13dfbd469c43b0e4dc033b65ed-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal\" (UID: \"a24aeb13dfbd469c43b0e4dc033b65ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal" Jul 28 17:10:16.962571 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:16.962510 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:17.042114 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:17.042082 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal" Jul 28 17:10:17.047234 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:17.047115 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-193.ec2.internal" Jul 28 17:10:17.063215 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:17.063184 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:17.163727 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:17.163663 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:18.595390 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:18.595370 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:18.695905 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:18.695850 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:18.737433 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:18.737397 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 28 17:10:18.737869 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:18.737577 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 17:10:18.797011 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:18.796984 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:18.817113 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:18.817073 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 17:05:17 +0000 UTC" deadline="2028-05-05 04:35:27.034704464 +0000 UTC" Jul 28 17:10:18.817113 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:18.817101 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15515h25m8.217607127s" Jul 28 17:10:18.897470 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:18.897404 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:18.938240 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:18.938200 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e753e1f8e3d27215843b4f99d76207a.slice/crio-7bfc62a54884771889d08891272ce9f5751a407155b62e41c91b587a3bd501a6 WatchSource:0}: Error finding container 7bfc62a54884771889d08891272ce9f5751a407155b62e41c91b587a3bd501a6: Status 404 returned error can't find the container with id 7bfc62a54884771889d08891272ce9f5751a407155b62e41c91b587a3bd501a6 Jul 28 17:10:18.938526 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:18.938498 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda24aeb13dfbd469c43b0e4dc033b65ed.slice/crio-9e62da9a174f181d46671184d5e2f4c2d571378c6b1510dc43e4f8adb7e1b5b9 WatchSource:0}: Error finding container 9e62da9a174f181d46671184d5e2f4c2d571378c6b1510dc43e4f8adb7e1b5b9: Status 404 returned error can't find the container with id 9e62da9a174f181d46671184d5e2f4c2d571378c6b1510dc43e4f8adb7e1b5b9 Jul 28 17:10:18.943060 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:18.943036 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:10:18.944106 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:18.944052 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal" event={"ID":"a24aeb13dfbd469c43b0e4dc033b65ed","Type":"ContainerStarted","Data":"9e62da9a174f181d46671184d5e2f4c2d571378c6b1510dc43e4f8adb7e1b5b9"} Jul 28 17:10:18.945435 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:18.945409 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-193.ec2.internal" event={"ID":"0e753e1f8e3d27215843b4f99d76207a","Type":"ContainerStarted","Data":"7bfc62a54884771889d08891272ce9f5751a407155b62e41c91b587a3bd501a6"} Jul 28 17:10:18.997869 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:18.997818 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:19.024652 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.024628 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:10:19.098696 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:19.098664 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:19.178888 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.178795 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:10:19.199435 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:19.199410 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-193.ec2.internal\" not found" Jul 28 17:10:19.248246 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.248215 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:10:19.299044 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.299005 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal" Jul 28 17:10:19.312377 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.312353 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 17:10:19.313816 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.313797 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-193.ec2.internal" Jul 28 17:10:19.324585 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.324560 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 17:10:19.778505 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.778476 2562 apiserver.go:52] "Watching apiserver" Jul 28 17:10:19.787632 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.787604 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 28 17:10:19.788897 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.788857 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal","openshift-multus/multus-additional-cni-plugins-zs8m6","openshift-multus/network-metrics-daemon-nq44j","openshift-network-diagnostics/network-check-target-mpsgl","kube-system/global-pull-secret-syncer-xn76m","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf","openshift-cluster-node-tuning-operator/tuned-mhgjl","openshift-multus/multus-xfvwc","openshift-network-operator/iptables-alerter-575t4","openshift-ovn-kubernetes/ovnkube-node-kxpf9","kube-system/konnectivity-agent-s4ggt","kube-system/kube-apiserver-proxy-ip-10-0-139-193.ec2.internal","openshift-dns/node-resolver-frb7f","openshift-image-registry/node-ca-w8nl6"] Jul 28 17:10:19.790664 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.790640 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-frb7f" Jul 28 17:10:19.791902 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.791859 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w8nl6" Jul 28 17:10:19.793094 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.793073 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.793803 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.793781 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 28 17:10:19.793803 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.793802 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 28 17:10:19.793967 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.793788 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-prhxd\"" Jul 28 17:10:19.794515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.794317 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:19.794515 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:19.794386 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:19.794811 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.794772 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 28 17:10:19.794811 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.794788 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 28 17:10:19.794965 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.794792 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m5cqz\"" Jul 28 17:10:19.794965 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.794776 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 28 17:10:19.795741 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.795633 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:19.795741 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:19.795693 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:19.796249 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.796012 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:10:19.796249 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.796051 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 28 17:10:19.796249 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.796076 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xh9rt\"" Jul 28 17:10:19.800359 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.798581 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.800359 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.799594 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 28 17:10:19.800359 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.800212 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.801399 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.801380 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 28 17:10:19.802353 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.801650 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-575t4" Jul 28 17:10:19.802353 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.801702 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 28 17:10:19.802353 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.801832 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wkzrz\"" Jul 28 17:10:19.802353 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.801983 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 28 17:10:19.802826 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.802804 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 28 17:10:19.803207 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.803188 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-r7w8h\"" Jul 28 17:10:19.803439 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.803424 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 28 17:10:19.803502 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.803481 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 28 17:10:19.803681 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.803667 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.804393 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.803668 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 28 17:10:19.805750 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.805721 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 28 17:10:19.807079 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.806274 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:10:19.807079 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.806572 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jz6bz\"" Jul 28 17:10:19.807079 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.806582 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 28 17:10:19.807698 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.807306 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s4ggt" Jul 28 17:10:19.807698 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.807372 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 28 17:10:19.808500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.808479 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 28 17:10:19.808597 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.808339 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6vjvp\"" Jul 28 17:10:19.808656 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.808597 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 28 17:10:19.808656 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.808639 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 28 17:10:19.808750 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.808602 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 28 17:10:19.809199 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.809182 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 28 17:10:19.809298 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.809281 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:19.809459 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:19.809432 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:19.811508 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.811252 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wd6mf\"" Jul 28 17:10:19.811644 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.811621 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:19.812058 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.812039 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 28 17:10:19.813249 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.812467 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813503 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-run-k8s-cni-cncf-io\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813534 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-sys-fs\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813561 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9df77\" (UniqueName: \"kubernetes.io/projected/7bf7f335-bb4f-46e6-b800-e81645e8ba71-kube-api-access-9df77\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813601 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-cnibin\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813621 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-sysctl-d\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813638 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-lib-modules\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813657 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vtwn\" (UniqueName: \"kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn\") pod \"network-check-target-mpsgl\" (UID: \"1ca5eb66-7139-4339-82cb-e0c5cabb18f4\") " pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813705 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/033fbb73-fda9-44a7-975b-edf75e79f4b1-multus-daemon-config\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813737 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28scl\" (UniqueName: \"kubernetes.io/projected/ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d-kube-api-access-28scl\") pod \"node-ca-w8nl6\" (UID: \"ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d\") " pod="openshift-image-registry/node-ca-w8nl6" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813775 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-sysconfig\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813798 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-var-lib-kubelet\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813816 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-host\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813856 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlqvv\" (UniqueName: \"kubernetes.io/projected/ab4df862-9a84-4853-8783-a56c85afb50d-kube-api-access-nlqvv\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813899 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4f49cd3a-ee50-43ee-a297-ca95c775c799-kubelet-config\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813924 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-device-dir\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.814231 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813947 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-var-lib-cni-multus\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813968 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab4df862-9a84-4853-8783-a56c85afb50d-tmp\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813968 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tzw6b\"" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.813999 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-socket-dir\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814035 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-var-lib-cni-bin\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814052 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d-serviceca\") pod \"node-ca-w8nl6\" (UID: \"ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d\") " pod="openshift-image-registry/node-ca-w8nl6" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814066 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-sysctl-conf\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814079 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-systemd\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814096 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814111 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-run\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814146 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/249db02c-8b5f-42ba-922e-871de558c0b1-hosts-file\") pod \"node-resolver-frb7f\" (UID: \"249db02c-8b5f-42ba-922e-871de558c0b1\") " pod="openshift-dns/node-resolver-frb7f" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814161 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbtwn\" (UniqueName: \"kubernetes.io/projected/249db02c-8b5f-42ba-922e-871de558c0b1-kube-api-access-gbtwn\") pod \"node-resolver-frb7f\" (UID: \"249db02c-8b5f-42ba-922e-871de558c0b1\") " pod="openshift-dns/node-resolver-frb7f" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814176 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814205 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-registration-dir\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814234 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-hostroot\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814258 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d-host\") pod \"node-ca-w8nl6\" (UID: \"ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d\") " pod="openshift-image-registry/node-ca-w8nl6" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814284 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-kubernetes\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814314 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:19.814909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814341 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-etc-selinux\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814364 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-multus-socket-dir-parent\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814388 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-run-netns\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814403 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-var-lib-kubelet\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814418 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-etc-kubernetes\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814439 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p744s\" (UniqueName: \"kubernetes.io/projected/033fbb73-fda9-44a7-975b-edf75e79f4b1-kube-api-access-p744s\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814491 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4f49cd3a-ee50-43ee-a297-ca95c775c799-dbus\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814529 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-multus-cni-dir\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814543 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-multus-conf-dir\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814573 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-run-multus-certs\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814608 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-modprobe-d\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814622 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-sys\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814638 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ab4df862-9a84-4853-8783-a56c85afb50d-etc-tuned\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814668 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814684 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/249db02c-8b5f-42ba-922e-871de558c0b1-tmp-dir\") pod \"node-resolver-frb7f\" (UID: \"249db02c-8b5f-42ba-922e-871de558c0b1\") " pod="openshift-dns/node-resolver-frb7f" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814711 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-system-cni-dir\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814726 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-os-release\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.815701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.814748 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/033fbb73-fda9-44a7-975b-edf75e79f4b1-cni-binary-copy\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.817868 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.817830 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 17:10:19.845324 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.845300 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xvnsq" Jul 28 17:10:19.853150 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.853127 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xvnsq" Jul 28 17:10:19.861512 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.861489 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:10:19.900406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.900378 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 28 17:10:19.915031 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915001 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab4df862-9a84-4853-8783-a56c85afb50d-tmp\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.915197 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915042 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-node-log\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.915197 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915065 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-log-socket\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.915197 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915087 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-socket-dir\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.915197 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915134 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-var-lib-cni-bin\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.915197 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915194 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-var-lib-cni-bin\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915214 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-sysctl-conf\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915252 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/83cde135-7db2-4834-bb14-f49744ba8df5-iptables-alerter-script\") pod \"iptables-alerter-575t4\" (UID: \"83cde135-7db2-4834-bb14-f49744ba8df5\") " pod="openshift-network-operator/iptables-alerter-575t4" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915284 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/249db02c-8b5f-42ba-922e-871de558c0b1-hosts-file\") pod \"node-resolver-frb7f\" (UID: \"249db02c-8b5f-42ba-922e-871de558c0b1\") " pod="openshift-dns/node-resolver-frb7f" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915307 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-socket-dir\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915310 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915354 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-registration-dir\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915370 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915381 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d-host\") pod \"node-ca-w8nl6\" (UID: \"ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d\") " pod="openshift-image-registry/node-ca-w8nl6" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915394 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-sysctl-conf\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915405 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-kubernetes\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915416 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/249db02c-8b5f-42ba-922e-871de558c0b1-hosts-file\") pod \"node-resolver-frb7f\" (UID: \"249db02c-8b5f-42ba-922e-871de558c0b1\") " pod="openshift-dns/node-resolver-frb7f" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915434 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-run-openvswitch\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915459 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d-host\") pod \"node-ca-w8nl6\" (UID: \"ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d\") " pod="openshift-image-registry/node-ca-w8nl6" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915451 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915462 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-ovnkube-script-lib\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915469 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-kubernetes\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.915527 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915422 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-registration-dir\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915520 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-multus-socket-dir-parent\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915568 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-run-netns\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915598 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-var-lib-kubelet\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915622 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p744s\" (UniqueName: \"kubernetes.io/projected/033fbb73-fda9-44a7-975b-edf75e79f4b1-kube-api-access-p744s\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915639 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-multus-socket-dir-parent\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915650 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915674 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-run-netns\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915680 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs\") pod \"network-metrics-daemon-nq44j\" (UID: \"64efaaa1-f45f-431e-a2d7-9b05a714971e\") " pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915683 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-var-lib-kubelet\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915732 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6c8m\" (UniqueName: \"kubernetes.io/projected/83cde135-7db2-4834-bb14-f49744ba8df5-kube-api-access-w6c8m\") pod \"iptables-alerter-575t4\" (UID: \"83cde135-7db2-4834-bb14-f49744ba8df5\") " pod="openshift-network-operator/iptables-alerter-575t4" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4f49cd3a-ee50-43ee-a297-ca95c775c799-dbus\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915809 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-multus-conf-dir\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915835 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-modprobe-d\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915859 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-sys\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915936 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ab4df862-9a84-4853-8783-a56c85afb50d-etc-tuned\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915959 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-modprobe-d\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.916406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915968 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-os-release\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915981 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-sys\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.915939 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4f49cd3a-ee50-43ee-a297-ca95c775c799-dbus\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916007 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/249db02c-8b5f-42ba-922e-871de558c0b1-tmp-dir\") pod \"node-resolver-frb7f\" (UID: \"249db02c-8b5f-42ba-922e-871de558c0b1\") " pod="openshift-dns/node-resolver-frb7f" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916036 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-system-cni-dir\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916064 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/033fbb73-fda9-44a7-975b-edf75e79f4b1-cni-binary-copy\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916090 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-run-k8s-cni-cncf-io\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916117 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gxth\" (UniqueName: \"kubernetes.io/projected/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-kube-api-access-5gxth\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916128 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-system-cni-dir\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916142 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916167 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-sys-fs\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916192 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9df77\" (UniqueName: \"kubernetes.io/projected/7bf7f335-bb4f-46e6-b800-e81645e8ba71-kube-api-access-9df77\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916219 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-cnibin\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916242 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-sysctl-d\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916266 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-slash\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916283 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-multus-conf-dir\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916288 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-etc-openvswitch\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.917163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916359 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-sys-fs\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-cni-bin\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916411 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjj7w\" (UniqueName: \"kubernetes.io/projected/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-kube-api-access-cjj7w\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916422 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/249db02c-8b5f-42ba-922e-871de558c0b1-tmp-dir\") pod \"node-resolver-frb7f\" (UID: \"249db02c-8b5f-42ba-922e-871de558c0b1\") " pod="openshift-dns/node-resolver-frb7f" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916440 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-var-lib-kubelet\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916473 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-host\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916550 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlqvv\" (UniqueName: \"kubernetes.io/projected/ab4df862-9a84-4853-8783-a56c85afb50d-kube-api-access-nlqvv\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916551 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-host\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916562 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-var-lib-kubelet\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916591 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-cnibin\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916616 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-run-k8s-cni-cncf-io\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916637 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-sysctl-d\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916699 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/033fbb73-fda9-44a7-975b-edf75e79f4b1-cni-binary-copy\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916721 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-run-netns\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916748 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-var-lib-openvswitch\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916771 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916793 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4f49cd3a-ee50-43ee-a297-ca95c775c799-kubelet-config\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:19.917925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916812 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-device-dir\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916831 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-var-lib-cni-multus\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916849 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-ovn-node-metrics-cert\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916860 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4f49cd3a-ee50-43ee-a297-ca95c775c799-kubelet-config\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916867 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4a894b9f-5c78-4ca0-9375-8b6465b5d5bc-konnectivity-ca\") pod \"konnectivity-agent-s4ggt\" (UID: \"4a894b9f-5c78-4ca0-9375-8b6465b5d5bc\") " pod="kube-system/konnectivity-agent-s4ggt" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916912 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d-serviceca\") pod \"node-ca-w8nl6\" (UID: \"ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d\") " pod="openshift-image-registry/node-ca-w8nl6" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916913 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-device-dir\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916963 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-var-lib-cni-multus\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916975 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-systemd\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.916994 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-run\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917012 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-cnibin\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917031 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-systemd\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbtwn\" (UniqueName: \"kubernetes.io/projected/249db02c-8b5f-42ba-922e-871de558c0b1-kube-api-access-gbtwn\") pod \"node-resolver-frb7f\" (UID: \"249db02c-8b5f-42ba-922e-871de558c0b1\") " pod="openshift-dns/node-resolver-frb7f" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917066 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-hostroot\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917088 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-systemd-units\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917112 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sslbl\" (UniqueName: \"kubernetes.io/projected/64efaaa1-f45f-431e-a2d7-9b05a714971e-kube-api-access-sslbl\") pod \"network-metrics-daemon-nq44j\" (UID: \"64efaaa1-f45f-431e-a2d7-9b05a714971e\") " pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917138 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4a894b9f-5c78-4ca0-9375-8b6465b5d5bc-agent-certs\") pod \"konnectivity-agent-s4ggt\" (UID: \"4a894b9f-5c78-4ca0-9375-8b6465b5d5bc\") " pod="kube-system/konnectivity-agent-s4ggt" Jul 28 17:10:19.918515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917152 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-hostroot\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917153 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-run\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917156 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917205 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-etc-selinux\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917225 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-etc-kubernetes\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917240 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-cni-netd\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:19.917240 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917265 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d-serviceca\") pod \"node-ca-w8nl6\" (UID: \"ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d\") " pod="openshift-image-registry/node-ca-w8nl6" Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917274 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-etc-kubernetes\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917315 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7bf7f335-bb4f-46e6-b800-e81645e8ba71-etc-selinux\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:19.917329 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret podName:4f49cd3a-ee50-43ee-a297-ca95c775c799 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:20.417299013 +0000 UTC m=+3.017751356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret") pod "global-pull-secret-syncer-xn76m" (UID: "4f49cd3a-ee50-43ee-a297-ca95c775c799") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917349 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83cde135-7db2-4834-bb14-f49744ba8df5-host-slash\") pod \"iptables-alerter-575t4\" (UID: \"83cde135-7db2-4834-bb14-f49744ba8df5\") " pod="openshift-network-operator/iptables-alerter-575t4" Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917369 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-multus-cni-dir\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917384 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-run-multus-certs\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917402 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-run-ovn-kubernetes\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917426 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-os-release\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917449 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-host-run-multus-certs\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.919167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917459 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-multus-cni-dir\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917479 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-ovnkube-config\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917511 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-env-overrides\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917513 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/033fbb73-fda9-44a7-975b-edf75e79f4b1-os-release\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917541 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-lib-modules\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917574 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-kubelet\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917595 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-run-systemd\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917617 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-lib-modules\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917620 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-run-ovn\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917656 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-system-cni-dir\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917683 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-cni-binary-copy\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917717 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917746 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vtwn\" (UniqueName: \"kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn\") pod \"network-check-target-mpsgl\" (UID: \"1ca5eb66-7139-4339-82cb-e0c5cabb18f4\") " pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917770 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/033fbb73-fda9-44a7-975b-edf75e79f4b1-multus-daemon-config\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917793 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28scl\" (UniqueName: \"kubernetes.io/projected/ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d-kube-api-access-28scl\") pod \"node-ca-w8nl6\" (UID: \"ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d\") " pod="openshift-image-registry/node-ca-w8nl6" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917816 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-sysconfig\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.917914 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ab4df862-9a84-4853-8783-a56c85afb50d-etc-sysconfig\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.919885 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.918306 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/033fbb73-fda9-44a7-975b-edf75e79f4b1-multus-daemon-config\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.920465 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.919337 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab4df862-9a84-4853-8783-a56c85afb50d-tmp\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.920465 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.919391 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ab4df862-9a84-4853-8783-a56c85afb50d-etc-tuned\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.924714 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.924691 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p744s\" (UniqueName: \"kubernetes.io/projected/033fbb73-fda9-44a7-975b-edf75e79f4b1-kube-api-access-p744s\") pod \"multus-xfvwc\" (UID: \"033fbb73-fda9-44a7-975b-edf75e79f4b1\") " pod="openshift-multus/multus-xfvwc" Jul 28 17:10:19.927093 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:19.927070 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:10:19.927215 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:19.927102 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:10:19.927215 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:19.927116 2562 projected.go:196] Error preparing data for projected volume kube-api-access-2vtwn for pod openshift-network-diagnostics/network-check-target-mpsgl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:19.927215 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:19.927180 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn podName:1ca5eb66-7139-4339-82cb-e0c5cabb18f4 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:20.427161958 +0000 UTC m=+3.027614297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2vtwn" (UniqueName: "kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn") pod "network-check-target-mpsgl" (UID: "1ca5eb66-7139-4339-82cb-e0c5cabb18f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:19.928041 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.928019 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9df77\" (UniqueName: \"kubernetes.io/projected/7bf7f335-bb4f-46e6-b800-e81645e8ba71-kube-api-access-9df77\") pod \"aws-ebs-csi-driver-node-7xmzf\" (UID: \"7bf7f335-bb4f-46e6-b800-e81645e8ba71\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:19.928157 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.928139 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbtwn\" (UniqueName: \"kubernetes.io/projected/249db02c-8b5f-42ba-922e-871de558c0b1-kube-api-access-gbtwn\") pod \"node-resolver-frb7f\" (UID: \"249db02c-8b5f-42ba-922e-871de558c0b1\") " pod="openshift-dns/node-resolver-frb7f" Jul 28 17:10:19.928910 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.928891 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlqvv\" (UniqueName: \"kubernetes.io/projected/ab4df862-9a84-4853-8783-a56c85afb50d-kube-api-access-nlqvv\") pod \"tuned-mhgjl\" (UID: \"ab4df862-9a84-4853-8783-a56c85afb50d\") " pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:19.930710 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:19.930643 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28scl\" (UniqueName: \"kubernetes.io/projected/ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d-kube-api-access-28scl\") pod \"node-ca-w8nl6\" (UID: \"ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d\") " pod="openshift-image-registry/node-ca-w8nl6" Jul 28 17:10:20.018994 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.018960 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-run-netns\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.018994 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.018997 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-var-lib-openvswitch\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.019211 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019021 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.019211 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019065 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-ovn-node-metrics-cert\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.019211 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019104 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4a894b9f-5c78-4ca0-9375-8b6465b5d5bc-konnectivity-ca\") pod \"konnectivity-agent-s4ggt\" (UID: \"4a894b9f-5c78-4ca0-9375-8b6465b5d5bc\") " pod="kube-system/konnectivity-agent-s4ggt" Jul 28 17:10:20.019211 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019068 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-run-netns\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.019211 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019151 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-var-lib-openvswitch\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.019211 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019202 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-cnibin\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.019490 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019226 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-systemd-units\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.019490 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019249 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sslbl\" (UniqueName: \"kubernetes.io/projected/64efaaa1-f45f-431e-a2d7-9b05a714971e-kube-api-access-sslbl\") pod \"network-metrics-daemon-nq44j\" (UID: \"64efaaa1-f45f-431e-a2d7-9b05a714971e\") " pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:20.019490 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019308 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-systemd-units\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.019490 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019310 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-cnibin\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.019490 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019337 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4a894b9f-5c78-4ca0-9375-8b6465b5d5bc-agent-certs\") pod \"konnectivity-agent-s4ggt\" (UID: \"4a894b9f-5c78-4ca0-9375-8b6465b5d5bc\") " pod="kube-system/konnectivity-agent-s4ggt" Jul 28 17:10:20.019490 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019389 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-cni-netd\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.019490 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019412 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83cde135-7db2-4834-bb14-f49744ba8df5-host-slash\") pod \"iptables-alerter-575t4\" (UID: \"83cde135-7db2-4834-bb14-f49744ba8df5\") " pod="openshift-network-operator/iptables-alerter-575t4" Jul 28 17:10:20.019490 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019438 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-run-ovn-kubernetes\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.019490 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019473 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-ovnkube-config\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.019490 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019476 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.019490 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019491 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-cni-netd\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-env-overrides\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019523 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-run-ovn-kubernetes\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019533 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-kubelet\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019556 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-run-systemd\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019563 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83cde135-7db2-4834-bb14-f49744ba8df5-host-slash\") pod \"iptables-alerter-575t4\" (UID: \"83cde135-7db2-4834-bb14-f49744ba8df5\") " pod="openshift-network-operator/iptables-alerter-575t4" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019579 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-run-ovn\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019601 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-system-cni-dir\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019628 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-cni-binary-copy\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019653 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019686 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4a894b9f-5c78-4ca0-9375-8b6465b5d5bc-konnectivity-ca\") pod \"konnectivity-agent-s4ggt\" (UID: \"4a894b9f-5c78-4ca0-9375-8b6465b5d5bc\") " pod="kube-system/konnectivity-agent-s4ggt" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019695 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-node-log\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019720 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-log-socket\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019746 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/83cde135-7db2-4834-bb14-f49744ba8df5-iptables-alerter-script\") pod \"iptables-alerter-575t4\" (UID: \"83cde135-7db2-4834-bb14-f49744ba8df5\") " pod="openshift-network-operator/iptables-alerter-575t4" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019749 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-run-ovn\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019775 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-run-openvswitch\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019795 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-kubelet\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020081 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019798 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-ovnkube-script-lib\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019839 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019867 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs\") pod \"network-metrics-daemon-nq44j\" (UID: \"64efaaa1-f45f-431e-a2d7-9b05a714971e\") " pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019912 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6c8m\" (UniqueName: \"kubernetes.io/projected/83cde135-7db2-4834-bb14-f49744ba8df5-kube-api-access-w6c8m\") pod \"iptables-alerter-575t4\" (UID: \"83cde135-7db2-4834-bb14-f49744ba8df5\") " pod="openshift-network-operator/iptables-alerter-575t4" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019920 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-run-systemd\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019940 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-env-overrides\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019944 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-os-release\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019959 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-node-log\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019991 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gxth\" (UniqueName: \"kubernetes.io/projected/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-kube-api-access-5gxth\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.020007 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-os-release\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.020018 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.020049 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-slash\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.020074 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-etc-openvswitch\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.020097 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-cni-bin\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.020121 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjj7w\" (UniqueName: \"kubernetes.io/projected/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-kube-api-access-cjj7w\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.020409 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-run-openvswitch\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.020492 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.020871 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.020556 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-system-cni-dir\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.021692 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.019990 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.021692 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.020571 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-etc-openvswitch\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.021692 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.020629 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-slash\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.021692 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:20.020634 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:20.021692 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.020662 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-host-cni-bin\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.021692 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:20.020700 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs podName:64efaaa1-f45f-431e-a2d7-9b05a714971e nodeName:}" failed. No retries permitted until 2026-07-28 17:10:20.520679089 +0000 UTC m=+3.121131413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs") pod "network-metrics-daemon-nq44j" (UID: "64efaaa1-f45f-431e-a2d7-9b05a714971e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:20.021692 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.021015 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-log-socket\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.021692 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.021133 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/83cde135-7db2-4834-bb14-f49744ba8df5-iptables-alerter-script\") pod \"iptables-alerter-575t4\" (UID: \"83cde135-7db2-4834-bb14-f49744ba8df5\") " pod="openshift-network-operator/iptables-alerter-575t4" Jul 28 17:10:20.021692 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.021291 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-ovnkube-config\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.021692 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.021364 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-cni-binary-copy\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.021692 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.021383 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.022090 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.021719 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-ovnkube-script-lib\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.022956 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.022931 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-ovn-node-metrics-cert\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.023072 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.022995 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4a894b9f-5c78-4ca0-9375-8b6465b5d5bc-agent-certs\") pod \"konnectivity-agent-s4ggt\" (UID: \"4a894b9f-5c78-4ca0-9375-8b6465b5d5bc\") " pod="kube-system/konnectivity-agent-s4ggt" Jul 28 17:10:20.031488 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.031401 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjj7w\" (UniqueName: \"kubernetes.io/projected/3f00b9d9-ad3e-4e58-8faa-da2eed76d38e-kube-api-access-cjj7w\") pod \"multus-additional-cni-plugins-zs8m6\" (UID: \"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e\") " pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.032185 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.032146 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sslbl\" (UniqueName: \"kubernetes.io/projected/64efaaa1-f45f-431e-a2d7-9b05a714971e-kube-api-access-sslbl\") pod \"network-metrics-daemon-nq44j\" (UID: \"64efaaa1-f45f-431e-a2d7-9b05a714971e\") " pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:20.033001 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.032980 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gxth\" (UniqueName: \"kubernetes.io/projected/9bc3a85a-7d3c-45dd-a991-1a0f8549dc57-kube-api-access-5gxth\") pod \"ovnkube-node-kxpf9\" (UID: \"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57\") " pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.033109 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.033059 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6c8m\" (UniqueName: \"kubernetes.io/projected/83cde135-7db2-4834-bb14-f49744ba8df5-kube-api-access-w6c8m\") pod \"iptables-alerter-575t4\" (UID: \"83cde135-7db2-4834-bb14-f49744ba8df5\") " pod="openshift-network-operator/iptables-alerter-575t4" Jul 28 17:10:20.103560 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.103521 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-frb7f" Jul 28 17:10:20.116939 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.116915 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w8nl6" Jul 28 17:10:20.125700 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.125670 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" Jul 28 17:10:20.130355 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.130331 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" Jul 28 17:10:20.136247 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.136227 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xfvwc" Jul 28 17:10:20.144417 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.144397 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-575t4" Jul 28 17:10:20.151316 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.151297 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:20.158712 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.158693 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s4ggt" Jul 28 17:10:20.165448 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.165423 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zs8m6" Jul 28 17:10:20.422979 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.422892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:20.423123 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:20.423051 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:20.423123 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:20.423113 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret podName:4f49cd3a-ee50-43ee-a297-ca95c775c799 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:21.423096397 +0000 UTC m=+4.023548721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret") pod "global-pull-secret-syncer-xn76m" (UID: "4f49cd3a-ee50-43ee-a297-ca95c775c799") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:20.524080 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.524045 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs\") pod \"network-metrics-daemon-nq44j\" (UID: \"64efaaa1-f45f-431e-a2d7-9b05a714971e\") " pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:20.524239 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.524139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vtwn\" (UniqueName: \"kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn\") pod \"network-check-target-mpsgl\" (UID: \"1ca5eb66-7139-4339-82cb-e0c5cabb18f4\") " pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:20.524239 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:20.524143 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:20.524239 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:20.524202 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs podName:64efaaa1-f45f-431e-a2d7-9b05a714971e nodeName:}" failed. No retries permitted until 2026-07-28 17:10:21.524185264 +0000 UTC m=+4.124637592 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs") pod "network-metrics-daemon-nq44j" (UID: "64efaaa1-f45f-431e-a2d7-9b05a714971e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:20.524356 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:20.524279 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:10:20.524356 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:20.524293 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:10:20.524356 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:20.524303 2562 projected.go:196] Error preparing data for projected volume kube-api-access-2vtwn for pod openshift-network-diagnostics/network-check-target-mpsgl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:20.524356 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:20.524346 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn podName:1ca5eb66-7139-4339-82cb-e0c5cabb18f4 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:21.524334278 +0000 UTC m=+4.124786601 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2vtwn" (UniqueName: "kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn") pod "network-check-target-mpsgl" (UID: "1ca5eb66-7139-4339-82cb-e0c5cabb18f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:20.775128 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:20.775054 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83cde135_7db2_4834_bb14_f49744ba8df5.slice/crio-ab07e75ab4d64bc3632db72e14b4bc309ba29d1381bc8f5f935cde0fb14b78ca WatchSource:0}: Error finding container ab07e75ab4d64bc3632db72e14b4bc309ba29d1381bc8f5f935cde0fb14b78ca: Status 404 returned error can't find the container with id ab07e75ab4d64bc3632db72e14b4bc309ba29d1381bc8f5f935cde0fb14b78ca Jul 28 17:10:20.777619 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:20.777577 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab4df862_9a84_4853_8783_a56c85afb50d.slice/crio-083e55925859eb9d201ec5c4ece6ace08838a2644a002b8ccfeef68f243c1a0f WatchSource:0}: Error finding container 083e55925859eb9d201ec5c4ece6ace08838a2644a002b8ccfeef68f243c1a0f: Status 404 returned error can't find the container with id 083e55925859eb9d201ec5c4ece6ace08838a2644a002b8ccfeef68f243c1a0f Jul 28 17:10:20.779702 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:20.779677 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef9a5eff_04c8_4af5_b00f_3dc2f5d0955d.slice/crio-5f8ae4b1e627cedef803495487d7c859f4ee611f2a77c418f311c82fb5188d0f WatchSource:0}: Error finding container 5f8ae4b1e627cedef803495487d7c859f4ee611f2a77c418f311c82fb5188d0f: Status 404 returned error can't find the container with id 5f8ae4b1e627cedef803495487d7c859f4ee611f2a77c418f311c82fb5188d0f Jul 28 17:10:20.780215 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:20.780176 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod033fbb73_fda9_44a7_975b_edf75e79f4b1.slice/crio-6b464032514929bf0fe2d92a179c81da1f5ff5a57dafaf2967a20c5031c47da5 WatchSource:0}: Error finding container 6b464032514929bf0fe2d92a179c81da1f5ff5a57dafaf2967a20c5031c47da5: Status 404 returned error can't find the container with id 6b464032514929bf0fe2d92a179c81da1f5ff5a57dafaf2967a20c5031c47da5 Jul 28 17:10:20.781486 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:20.781461 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc3a85a_7d3c_45dd_a991_1a0f8549dc57.slice/crio-46c9e25f901cd0d4af35d9bcd042e3a89802d2071ba72e4266c793d79933cc48 WatchSource:0}: Error finding container 46c9e25f901cd0d4af35d9bcd042e3a89802d2071ba72e4266c793d79933cc48: Status 404 returned error can't find the container with id 46c9e25f901cd0d4af35d9bcd042e3a89802d2071ba72e4266c793d79933cc48 Jul 28 17:10:20.783100 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:20.782972 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a894b9f_5c78_4ca0_9375_8b6465b5d5bc.slice/crio-1fc32da01545a7eb159997ca1fac0c1eb70dbcc2845f44a19c6a48bd83d12149 WatchSource:0}: Error finding container 1fc32da01545a7eb159997ca1fac0c1eb70dbcc2845f44a19c6a48bd83d12149: Status 404 returned error can't find the container with id 1fc32da01545a7eb159997ca1fac0c1eb70dbcc2845f44a19c6a48bd83d12149 Jul 28 17:10:20.784037 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:20.784016 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod249db02c_8b5f_42ba_922e_871de558c0b1.slice/crio-47cd11033c78dd74b60556a25c0b9d74dbbdd1ab2fd4b2612928cc559acb26df WatchSource:0}: Error finding container 47cd11033c78dd74b60556a25c0b9d74dbbdd1ab2fd4b2612928cc559acb26df: Status 404 returned error can't find the container with id 47cd11033c78dd74b60556a25c0b9d74dbbdd1ab2fd4b2612928cc559acb26df Jul 28 17:10:20.785736 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:20.785672 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f00b9d9_ad3e_4e58_8faa_da2eed76d38e.slice/crio-cafe972390086edd6ebb63588d2e3963a1cd5059cdd96875fe3d0805f4bd17af WatchSource:0}: Error finding container cafe972390086edd6ebb63588d2e3963a1cd5059cdd96875fe3d0805f4bd17af: Status 404 returned error can't find the container with id cafe972390086edd6ebb63588d2e3963a1cd5059cdd96875fe3d0805f4bd17af Jul 28 17:10:20.787146 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:20.787126 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bf7f335_bb4f_46e6_b800_e81645e8ba71.slice/crio-4bbf42dcacf862ef74079c8b592bba101bf5d4adfa7542f36de5b9d16501354a WatchSource:0}: Error finding container 4bbf42dcacf862ef74079c8b592bba101bf5d4adfa7542f36de5b9d16501354a: Status 404 returned error can't find the container with id 4bbf42dcacf862ef74079c8b592bba101bf5d4adfa7542f36de5b9d16501354a Jul 28 17:10:20.855267 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.855202 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:05:19 +0000 UTC" deadline="2028-05-15 12:25:11.370837026 +0000 UTC" Jul 28 17:10:20.855267 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.855267 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15763h14m50.515573987s" Jul 28 17:10:20.940555 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.940521 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:20.940711 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:20.940662 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:20.949971 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.949941 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-575t4" event={"ID":"83cde135-7db2-4834-bb14-f49744ba8df5","Type":"ContainerStarted","Data":"ab07e75ab4d64bc3632db72e14b4bc309ba29d1381bc8f5f935cde0fb14b78ca"} Jul 28 17:10:20.951812 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.951785 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-193.ec2.internal" event={"ID":"0e753e1f8e3d27215843b4f99d76207a","Type":"ContainerStarted","Data":"895f27c026a9eb9987173ae6e7d1a49e4f8489d881353bb560a2b8b87c2057fe"} Jul 28 17:10:20.953480 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.953457 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s4ggt" event={"ID":"4a894b9f-5c78-4ca0-9375-8b6465b5d5bc","Type":"ContainerStarted","Data":"1fc32da01545a7eb159997ca1fac0c1eb70dbcc2845f44a19c6a48bd83d12149"} Jul 28 17:10:20.954666 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.954637 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w8nl6" event={"ID":"ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d","Type":"ContainerStarted","Data":"5f8ae4b1e627cedef803495487d7c859f4ee611f2a77c418f311c82fb5188d0f"} Jul 28 17:10:20.955678 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.955651 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" event={"ID":"7bf7f335-bb4f-46e6-b800-e81645e8ba71","Type":"ContainerStarted","Data":"4bbf42dcacf862ef74079c8b592bba101bf5d4adfa7542f36de5b9d16501354a"} Jul 28 17:10:20.956531 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.956511 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs8m6" event={"ID":"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e","Type":"ContainerStarted","Data":"cafe972390086edd6ebb63588d2e3963a1cd5059cdd96875fe3d0805f4bd17af"} Jul 28 17:10:20.957423 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.957399 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-frb7f" event={"ID":"249db02c-8b5f-42ba-922e-871de558c0b1","Type":"ContainerStarted","Data":"47cd11033c78dd74b60556a25c0b9d74dbbdd1ab2fd4b2612928cc559acb26df"} Jul 28 17:10:20.958477 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.958460 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" event={"ID":"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57","Type":"ContainerStarted","Data":"46c9e25f901cd0d4af35d9bcd042e3a89802d2071ba72e4266c793d79933cc48"} Jul 28 17:10:20.959417 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.959396 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xfvwc" event={"ID":"033fbb73-fda9-44a7-975b-edf75e79f4b1","Type":"ContainerStarted","Data":"6b464032514929bf0fe2d92a179c81da1f5ff5a57dafaf2967a20c5031c47da5"} Jul 28 17:10:20.960328 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:20.960305 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" event={"ID":"ab4df862-9a84-4853-8783-a56c85afb50d","Type":"ContainerStarted","Data":"083e55925859eb9d201ec5c4ece6ace08838a2644a002b8ccfeef68f243c1a0f"} Jul 28 17:10:21.433558 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:21.432766 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:21.433558 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:21.433019 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:21.433558 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:21.433147 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret podName:4f49cd3a-ee50-43ee-a297-ca95c775c799 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:23.433126425 +0000 UTC m=+6.033578771 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret") pod "global-pull-secret-syncer-xn76m" (UID: "4f49cd3a-ee50-43ee-a297-ca95c775c799") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:21.533973 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:21.533920 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vtwn\" (UniqueName: \"kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn\") pod \"network-check-target-mpsgl\" (UID: \"1ca5eb66-7139-4339-82cb-e0c5cabb18f4\") " pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:21.534147 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:21.534001 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs\") pod \"network-metrics-daemon-nq44j\" (UID: \"64efaaa1-f45f-431e-a2d7-9b05a714971e\") " pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:21.534232 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:21.534191 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:21.534282 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:21.534260 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs podName:64efaaa1-f45f-431e-a2d7-9b05a714971e nodeName:}" failed. No retries permitted until 2026-07-28 17:10:23.534239021 +0000 UTC m=+6.134691348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs") pod "network-metrics-daemon-nq44j" (UID: "64efaaa1-f45f-431e-a2d7-9b05a714971e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:21.534706 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:21.534684 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:10:21.534706 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:21.534707 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:10:21.534849 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:21.534721 2562 projected.go:196] Error preparing data for projected volume kube-api-access-2vtwn for pod openshift-network-diagnostics/network-check-target-mpsgl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:21.534849 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:21.534774 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn podName:1ca5eb66-7139-4339-82cb-e0c5cabb18f4 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:23.534756489 +0000 UTC m=+6.135208813 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2vtwn" (UniqueName: "kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn") pod "network-check-target-mpsgl" (UID: "1ca5eb66-7139-4339-82cb-e0c5cabb18f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:21.856947 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:21.856483 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:05:19 +0000 UTC" deadline="2028-03-25 23:54:55.478456832 +0000 UTC" Jul 28 17:10:21.856947 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:21.856525 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14550h44m33.621936805s" Jul 28 17:10:21.941920 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:21.941887 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:21.942088 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:21.942054 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:21.942172 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:21.942144 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:21.942528 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:21.942507 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:21.969697 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:21.969641 2562 generic.go:356] "Generic (PLEG): container finished" podID="a24aeb13dfbd469c43b0e4dc033b65ed" containerID="2d17e6491562d04f35daaef8a2465f0de918b89d7454dad3906437af4906fb72" exitCode=0 Jul 28 17:10:21.970646 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:21.970544 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal" event={"ID":"a24aeb13dfbd469c43b0e4dc033b65ed","Type":"ContainerDied","Data":"2d17e6491562d04f35daaef8a2465f0de918b89d7454dad3906437af4906fb72"} Jul 28 17:10:21.990752 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:21.989948 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-193.ec2.internal" podStartSLOduration=2.989927065 podStartE2EDuration="2.989927065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:20.967798159 +0000 UTC m=+3.568250505" watchObservedRunningTime="2026-07-28 17:10:21.989927065 +0000 UTC m=+4.590379412" Jul 28 17:10:22.941257 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:22.941175 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:22.941701 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:22.941323 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:22.978001 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:22.977961 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal" event={"ID":"a24aeb13dfbd469c43b0e4dc033b65ed","Type":"ContainerStarted","Data":"d385eede80c48643e605299dbeb1a11e9de3e4e08a41920668aa12bed1138840"} Jul 28 17:10:23.457710 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:23.457669 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:23.457919 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:23.457820 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:23.457919 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:23.457907 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret podName:4f49cd3a-ee50-43ee-a297-ca95c775c799 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:27.457865081 +0000 UTC m=+10.058317408 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret") pod "global-pull-secret-syncer-xn76m" (UID: "4f49cd3a-ee50-43ee-a297-ca95c775c799") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:23.559630 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:23.559029 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vtwn\" (UniqueName: \"kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn\") pod \"network-check-target-mpsgl\" (UID: \"1ca5eb66-7139-4339-82cb-e0c5cabb18f4\") " pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:23.559630 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:23.559088 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs\") pod \"network-metrics-daemon-nq44j\" (UID: \"64efaaa1-f45f-431e-a2d7-9b05a714971e\") " pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:23.559630 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:23.559221 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:23.559630 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:23.559225 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:10:23.559630 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:23.559250 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:10:23.559630 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:23.559263 2562 projected.go:196] Error preparing data for projected volume kube-api-access-2vtwn for pod openshift-network-diagnostics/network-check-target-mpsgl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:23.559630 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:23.559281 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs podName:64efaaa1-f45f-431e-a2d7-9b05a714971e nodeName:}" failed. No retries permitted until 2026-07-28 17:10:27.559263001 +0000 UTC m=+10.159715327 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs") pod "network-metrics-daemon-nq44j" (UID: "64efaaa1-f45f-431e-a2d7-9b05a714971e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:23.559630 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:23.559316 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn podName:1ca5eb66-7139-4339-82cb-e0c5cabb18f4 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:27.559300049 +0000 UTC m=+10.159752375 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2vtwn" (UniqueName: "kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn") pod "network-check-target-mpsgl" (UID: "1ca5eb66-7139-4339-82cb-e0c5cabb18f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:23.943652 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:23.942887 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:23.943652 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:23.943039 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:23.943652 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:23.943451 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:23.943652 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:23.943543 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:24.940408 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:24.940370 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:24.940635 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:24.940539 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:25.942024 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:25.941990 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:25.942454 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:25.942033 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:25.942454 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:25.942140 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:25.942454 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:25.942264 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:26.941317 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:26.940742 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:26.941317 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:26.940893 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:27.496734 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:27.496661 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:27.497194 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:27.496832 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:27.497194 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:27.496915 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret podName:4f49cd3a-ee50-43ee-a297-ca95c775c799 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:35.496894688 +0000 UTC m=+18.097347016 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret") pod "global-pull-secret-syncer-xn76m" (UID: "4f49cd3a-ee50-43ee-a297-ca95c775c799") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:27.597199 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:27.597156 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs\") pod \"network-metrics-daemon-nq44j\" (UID: \"64efaaa1-f45f-431e-a2d7-9b05a714971e\") " pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:27.597395 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:27.597261 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vtwn\" (UniqueName: \"kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn\") pod \"network-check-target-mpsgl\" (UID: \"1ca5eb66-7139-4339-82cb-e0c5cabb18f4\") " pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:27.597395 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:27.597377 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:27.597395 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:27.597393 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:10:27.597572 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:27.597409 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:10:27.597572 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:27.597422 2562 projected.go:196] Error preparing data for projected volume kube-api-access-2vtwn for pod openshift-network-diagnostics/network-check-target-mpsgl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:27.597572 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:27.597452 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs podName:64efaaa1-f45f-431e-a2d7-9b05a714971e nodeName:}" failed. No retries permitted until 2026-07-28 17:10:35.597432539 +0000 UTC m=+18.197884876 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs") pod "network-metrics-daemon-nq44j" (UID: "64efaaa1-f45f-431e-a2d7-9b05a714971e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:27.597572 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:27.597473 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn podName:1ca5eb66-7139-4339-82cb-e0c5cabb18f4 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:35.59746185 +0000 UTC m=+18.197914179 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2vtwn" (UniqueName: "kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn") pod "network-check-target-mpsgl" (UID: "1ca5eb66-7139-4339-82cb-e0c5cabb18f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:27.942153 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:27.942071 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:27.942309 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:27.942200 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:27.942309 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:27.942257 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:27.942405 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:27.942381 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:28.941026 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:28.940990 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:28.941487 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:28.941126 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:29.940719 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:29.940682 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:29.940932 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:29.940824 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:29.941308 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:29.941289 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:29.941653 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:29.941367 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:30.940126 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:30.940093 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:30.940316 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:30.940227 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:31.941631 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:31.941118 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:31.941631 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:31.941157 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:31.941631 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:31.941273 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:31.941631 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:31.941369 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:32.940237 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:32.940202 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:32.940424 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:32.940336 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:33.943456 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:33.942988 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:33.943456 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:33.943145 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:33.943456 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:33.943175 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:33.943456 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:33.943264 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:34.940756 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:34.940711 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:34.940995 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:34.940892 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:35.554650 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:35.554612 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:35.555182 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:35.554763 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:35.555182 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:35.554825 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret podName:4f49cd3a-ee50-43ee-a297-ca95c775c799 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:51.554808437 +0000 UTC m=+34.155260777 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret") pod "global-pull-secret-syncer-xn76m" (UID: "4f49cd3a-ee50-43ee-a297-ca95c775c799") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:35.655310 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:35.655269 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vtwn\" (UniqueName: \"kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn\") pod \"network-check-target-mpsgl\" (UID: \"1ca5eb66-7139-4339-82cb-e0c5cabb18f4\") " pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:35.655310 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:35.655320 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs\") pod \"network-metrics-daemon-nq44j\" (UID: \"64efaaa1-f45f-431e-a2d7-9b05a714971e\") " pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:35.655530 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:35.655417 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:35.655530 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:35.655442 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:10:35.655530 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:35.655469 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:10:35.655530 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:35.655482 2562 projected.go:196] Error preparing data for projected volume kube-api-access-2vtwn for pod openshift-network-diagnostics/network-check-target-mpsgl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:35.655530 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:35.655482 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs podName:64efaaa1-f45f-431e-a2d7-9b05a714971e nodeName:}" failed. No retries permitted until 2026-07-28 17:10:51.655466724 +0000 UTC m=+34.255919071 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs") pod "network-metrics-daemon-nq44j" (UID: "64efaaa1-f45f-431e-a2d7-9b05a714971e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:35.655802 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:35.655556 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn podName:1ca5eb66-7139-4339-82cb-e0c5cabb18f4 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:51.655538955 +0000 UTC m=+34.255991286 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2vtwn" (UniqueName: "kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn") pod "network-check-target-mpsgl" (UID: "1ca5eb66-7139-4339-82cb-e0c5cabb18f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:35.940554 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:35.940464 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:35.940720 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:35.940464 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:35.940720 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:35.940616 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:35.940720 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:35.940672 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:36.940258 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:36.940218 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:36.940656 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:36.940369 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:37.941588 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:37.941512 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:37.942109 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:37.941625 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:37.942109 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:37.941658 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:37.942109 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:37.941762 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:38.940362 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:38.940320 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:38.940566 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:38.940453 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:39.941509 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:39.941082 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:39.942246 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:39.941130 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:39.942246 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:39.941643 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:39.942246 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:39.941668 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:40.015189 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.015097 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" event={"ID":"7bf7f335-bb4f-46e6-b800-e81645e8ba71","Type":"ContainerStarted","Data":"a1352b12540582cec8019d95d2b073fa1ce236d60760c370423d98f68baadebe"} Jul 28 17:10:40.016807 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.016777 2562 generic.go:356] "Generic (PLEG): container finished" podID="3f00b9d9-ad3e-4e58-8faa-da2eed76d38e" containerID="5591a3dea0df4202c1998fe4bd4cf843ae8e7dfabed57116498327af2c8aff77" exitCode=0 Jul 28 17:10:40.016969 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.016888 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs8m6" event={"ID":"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e","Type":"ContainerDied","Data":"5591a3dea0df4202c1998fe4bd4cf843ae8e7dfabed57116498327af2c8aff77"} Jul 28 17:10:40.018515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.018485 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-frb7f" event={"ID":"249db02c-8b5f-42ba-922e-871de558c0b1","Type":"ContainerStarted","Data":"44b52460bfbc4cfa06c42f86c42acc8f6d945dc331458bb3ff9ce4d2f2f201a1"} Jul 28 17:10:40.022035 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.022014 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" event={"ID":"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57","Type":"ContainerStarted","Data":"d88a37d41b52d05e76b4aeb772662aee7695fb501ac36245ab1037cd4b146ce1"} Jul 28 17:10:40.022128 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.022087 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" event={"ID":"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57","Type":"ContainerStarted","Data":"131ae2ce1670c97b7a238690fc75c6bb67326d910af641db0b38c9aa713e2e96"} Jul 28 17:10:40.022185 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.022150 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" event={"ID":"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57","Type":"ContainerStarted","Data":"6be16438eab35d0dff310b0f3be55bd7c205f413d55ecd5997b090ead6fd9a08"} Jul 28 17:10:40.022237 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.022200 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" event={"ID":"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57","Type":"ContainerStarted","Data":"b7490f5e0c1c95d649444de4033c797475845888fc753081db4b372142ac1c46"} Jul 28 17:10:40.022303 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.022248 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" event={"ID":"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57","Type":"ContainerStarted","Data":"fb65b1b6b8c38fdd396113ba028e5121ff929c0f5c2a83e37b9ec6bedb2ded35"} Jul 28 17:10:40.022303 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.022295 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" event={"ID":"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57","Type":"ContainerStarted","Data":"3235a432bc930159302050d40c39232cbf923eaa659fa2672b7a594fd5eda561"} Jul 28 17:10:40.023888 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.023851 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xfvwc" event={"ID":"033fbb73-fda9-44a7-975b-edf75e79f4b1","Type":"ContainerStarted","Data":"cdce7be875690ce1cdb89a079b4d6692898ba4afeda7a142ab32fadbb339251c"} Jul 28 17:10:40.026073 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.026052 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" event={"ID":"ab4df862-9a84-4853-8783-a56c85afb50d","Type":"ContainerStarted","Data":"008602c24b19021ebb7cb35853a082c0dd4a2b6ce8a9b0e6e63ece9759a7b561"} Jul 28 17:10:40.027973 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.027945 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s4ggt" event={"ID":"4a894b9f-5c78-4ca0-9375-8b6465b5d5bc","Type":"ContainerStarted","Data":"7a16ecc3e934c0c47a425c5cd3114e00fd8e614bacab593f81f19b617ecbe764"} Jul 28 17:10:40.031512 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.031487 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w8nl6" event={"ID":"ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d","Type":"ContainerStarted","Data":"ea09c61644334c68dae14351497bec2d22f54c4ce33949ccf08f1b4e1c456e71"} Jul 28 17:10:40.044673 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.044627 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-193.ec2.internal" podStartSLOduration=21.044613952 podStartE2EDuration="21.044613952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:22.998578755 +0000 UTC m=+5.599031111" watchObservedRunningTime="2026-07-28 17:10:40.044613952 +0000 UTC m=+22.645066297" Jul 28 17:10:40.070068 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.070021 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-frb7f" podStartSLOduration=4.721746658 podStartE2EDuration="23.070006548s" totalImagesPullingTime="18.34825989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:40.069469843 +0000 UTC m=+22.669922188" watchObservedRunningTime="2026-07-28 17:10:40.070006548 +0000 UTC m=+22.670458893" Jul 28 17:10:40.112343 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.112299 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mhgjl" podStartSLOduration=3.7222882999999998 podStartE2EDuration="22.112284346s" totalImagesPullingTime="18.389996046s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:40.112082536 +0000 UTC m=+22.712534881" watchObservedRunningTime="2026-07-28 17:10:40.112284346 +0000 UTC m=+22.712736693" Jul 28 17:10:40.112714 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.112689 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w8nl6" podStartSLOduration=4.762620181 podStartE2EDuration="23.112684027s" totalImagesPullingTime="18.350063846s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:40.087289901 +0000 UTC m=+22.687742260" watchObservedRunningTime="2026-07-28 17:10:40.112684027 +0000 UTC m=+22.713136373" Jul 28 17:10:40.131714 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.131673 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xfvwc" podStartSLOduration=3.7066273560000003 podStartE2EDuration="22.131657281s" totalImagesPullingTime="18.425029925s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:40.131262038 +0000 UTC m=+22.731714383" watchObservedRunningTime="2026-07-28 17:10:40.131657281 +0000 UTC m=+22.732109627" Jul 28 17:10:40.150940 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.150868 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-s4ggt" podStartSLOduration=7.88260837 podStartE2EDuration="22.150849556s" totalImagesPullingTime="14.268241186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:40.150110765 +0000 UTC m=+22.750563112" watchObservedRunningTime="2026-07-28 17:10:40.150849556 +0000 UTC m=+22.751301903" Jul 28 17:10:40.498482 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.498259 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 28 17:10:40.866086 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.865969 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T17:10:40.498275911Z","UUID":"54f015a2-786b-470f-8e08-fa47af4e0bcd","Handler":null,"Name":"","Endpoint":""} Jul 28 17:10:40.868083 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.868057 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 28 17:10:40.868279 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.868107 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 28 17:10:40.940465 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:40.940429 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:40.940631 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:40.940580 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:41.035911 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:41.035852 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-575t4" event={"ID":"83cde135-7db2-4834-bb14-f49744ba8df5","Type":"ContainerStarted","Data":"a255a4cf582d3867b69ba46d38a7982512bbf1707c03bbb2f8d457223c2d3917"} Jul 28 17:10:41.038799 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:41.038770 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" event={"ID":"7bf7f335-bb4f-46e6-b800-e81645e8ba71","Type":"ContainerStarted","Data":"44bd04688c30fbfb6f3d8c9d5969cde99671a89808985d05c8762fad5c9e6f70"} Jul 28 17:10:41.065089 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:41.065030 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-575t4" podStartSLOduration=4.710124877 podStartE2EDuration="23.065008099s" totalImagesPullingTime="18.354883222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:41.064080565 +0000 UTC m=+23.664532911" watchObservedRunningTime="2026-07-28 17:10:41.065008099 +0000 UTC m=+23.665460449" Jul 28 17:10:41.921286 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:41.921199 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-s4ggt" Jul 28 17:10:41.922000 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:41.921977 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-s4ggt" Jul 28 17:10:41.940596 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:41.940317 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:41.940596 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:41.940337 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:41.940596 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:41.940422 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:41.940596 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:41.940503 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:42.043698 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:42.043665 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" event={"ID":"7bf7f335-bb4f-46e6-b800-e81645e8ba71","Type":"ContainerStarted","Data":"91cbe62180fe7c88a5aaa2d6ea227ded8611adbede696ddff0ea58d0074f08b5"} Jul 28 17:10:42.048077 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:42.047619 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" event={"ID":"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57","Type":"ContainerStarted","Data":"65820b203234871c98fd5119b8cfe447e5771f61a292cf63091c5cbcc7d0fea6"} Jul 28 17:10:42.048077 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:42.048044 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-s4ggt" Jul 28 17:10:42.049324 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:42.049306 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-s4ggt" Jul 28 17:10:42.065835 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:42.065776 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7xmzf" podStartSLOduration=3.696009018 podStartE2EDuration="24.065755248s" totalImagesPullingTime="20.36974623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:42.065330289 +0000 UTC m=+24.665782635" watchObservedRunningTime="2026-07-28 17:10:42.065755248 +0000 UTC m=+24.666207661" Jul 28 17:10:42.940352 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:42.940316 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:42.940630 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:42.940445 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:43.940654 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:43.940631 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:43.940654 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:43.940650 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:43.941189 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:43.940761 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:43.941189 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:43.940902 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:44.941000 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:44.940908 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:44.941736 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:44.941043 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:45.056329 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:45.056287 2562 generic.go:356] "Generic (PLEG): container finished" podID="3f00b9d9-ad3e-4e58-8faa-da2eed76d38e" containerID="51df42ad514f2d22ae896bd8315080f1c83f9bbe5c6f52340c221ff5f3481cd4" exitCode=0 Jul 28 17:10:45.056487 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:45.056324 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs8m6" event={"ID":"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e","Type":"ContainerDied","Data":"51df42ad514f2d22ae896bd8315080f1c83f9bbe5c6f52340c221ff5f3481cd4"} Jul 28 17:10:45.060010 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:45.059969 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" event={"ID":"9bc3a85a-7d3c-45dd-a991-1a0f8549dc57","Type":"ContainerStarted","Data":"a58bb90186d1694d83cbfb2db0b852cb875cd67f392493ac2a0db1e9b430c861"} Jul 28 17:10:45.060181 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:45.060165 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:45.060238 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:45.060216 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:45.075248 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:45.075228 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:45.131307 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:45.131255 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" podStartSLOduration=8.700405636 podStartE2EDuration="27.13123878s" totalImagesPullingTime="18.430833144s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:45.129580674 +0000 UTC m=+27.730033023" watchObservedRunningTime="2026-07-28 17:10:45.13123878 +0000 UTC m=+27.731691126" Jul 28 17:10:45.944062 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:45.943868 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:45.944576 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:45.943870 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:45.944576 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:45.944191 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:45.944576 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:45.944295 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:46.062376 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:46.062350 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:46.076411 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:46.076389 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:10:46.160382 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:46.160356 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xn76m"] Jul 28 17:10:46.160520 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:46.160459 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:46.160591 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:46.160568 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:46.163125 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:46.163100 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mpsgl"] Jul 28 17:10:46.163274 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:46.163191 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:46.163337 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:46.163292 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:46.163713 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:46.163688 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nq44j"] Jul 28 17:10:46.163818 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:46.163783 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:46.163903 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:46.163869 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:47.066686 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:47.066608 2562 generic.go:356] "Generic (PLEG): container finished" podID="3f00b9d9-ad3e-4e58-8faa-da2eed76d38e" containerID="48be5b5b9e617e896d32e46c45d88a68c1ce15ef73f42e1eb8ee82346740768a" exitCode=0 Jul 28 17:10:47.067083 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:47.066700 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs8m6" event={"ID":"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e","Type":"ContainerDied","Data":"48be5b5b9e617e896d32e46c45d88a68c1ce15ef73f42e1eb8ee82346740768a"} Jul 28 17:10:47.943321 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:47.943285 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:47.943321 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:47.943311 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:47.943507 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:47.943325 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:47.943507 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:47.943399 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:47.943507 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:47.943441 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:47.943605 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:47.943519 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:49.074617 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:49.074387 2562 generic.go:356] "Generic (PLEG): container finished" podID="3f00b9d9-ad3e-4e58-8faa-da2eed76d38e" containerID="6eea9b79aacb8d62a30197b8ef6b104717819e8ef4ab60564201c26a21a93261" exitCode=0 Jul 28 17:10:49.074617 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:49.074473 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs8m6" event={"ID":"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e","Type":"ContainerDied","Data":"6eea9b79aacb8d62a30197b8ef6b104717819e8ef4ab60564201c26a21a93261"} Jul 28 17:10:49.940239 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:49.940207 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:49.940454 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:49.940207 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:49.940454 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:49.940350 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:49.940454 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:49.940349 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:49.940660 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:49.940471 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:49.940660 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:49.940561 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:51.567989 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:51.567953 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:51.568546 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:51.568129 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:51.568546 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:51.568196 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret podName:4f49cd3a-ee50-43ee-a297-ca95c775c799 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:23.568176529 +0000 UTC m=+66.168628853 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret") pod "global-pull-secret-syncer-xn76m" (UID: "4f49cd3a-ee50-43ee-a297-ca95c775c799") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:51.669026 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:51.668993 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vtwn\" (UniqueName: \"kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn\") pod \"network-check-target-mpsgl\" (UID: \"1ca5eb66-7139-4339-82cb-e0c5cabb18f4\") " pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:51.669188 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:51.669049 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs\") pod \"network-metrics-daemon-nq44j\" (UID: \"64efaaa1-f45f-431e-a2d7-9b05a714971e\") " pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:51.669188 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:51.669162 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:10:51.669188 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:51.669180 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:10:51.669188 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:51.669191 2562 projected.go:196] Error preparing data for projected volume kube-api-access-2vtwn for pod openshift-network-diagnostics/network-check-target-mpsgl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:51.669352 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:51.669202 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:51.669352 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:51.669243 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn podName:1ca5eb66-7139-4339-82cb-e0c5cabb18f4 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:23.669228947 +0000 UTC m=+66.269681272 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2vtwn" (UniqueName: "kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn") pod "network-check-target-mpsgl" (UID: "1ca5eb66-7139-4339-82cb-e0c5cabb18f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:51.669352 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:51.669259 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs podName:64efaaa1-f45f-431e-a2d7-9b05a714971e nodeName:}" failed. No retries permitted until 2026-07-28 17:11:23.669252347 +0000 UTC m=+66.269704670 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs") pod "network-metrics-daemon-nq44j" (UID: "64efaaa1-f45f-431e-a2d7-9b05a714971e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:51.940411 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:51.940324 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:51.940411 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:51.940349 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:51.940411 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:51.940324 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:51.940657 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:51.940467 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpsgl" podUID="1ca5eb66-7139-4339-82cb-e0c5cabb18f4" Jul 28 17:10:51.940657 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:51.940533 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn76m" podUID="4f49cd3a-ee50-43ee-a297-ca95c775c799" Jul 28 17:10:51.940657 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:51.940583 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nq44j" podUID="64efaaa1-f45f-431e-a2d7-9b05a714971e" Jul 28 17:10:52.680391 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.680356 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-193.ec2.internal" event="NodeReady" Jul 28 17:10:52.680819 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.680519 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 28 17:10:52.721371 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.721328 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b54dc8599-qn4bf"] Jul 28 17:10:52.723432 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.723406 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.726990 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.726968 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-r5xqg\"" Jul 28 17:10:52.727290 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.727272 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 28 17:10:52.727902 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.727863 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 28 17:10:52.728262 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.728240 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 28 17:10:52.733315 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.733296 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 28 17:10:52.749087 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.749063 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qqpxl"] Jul 28 17:10:52.751056 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.751034 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9m9d6"] Jul 28 17:10:52.751177 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.751161 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qqpxl" Jul 28 17:10:52.753200 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.753185 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9m9d6" Jul 28 17:10:52.756914 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.756896 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b54dc8599-qn4bf"] Jul 28 17:10:52.758114 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.757996 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 28 17:10:52.758114 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.758011 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 28 17:10:52.758114 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.758052 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 28 17:10:52.758308 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.758218 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 28 17:10:52.758366 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.758330 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 28 17:10:52.758420 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.758368 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hqvdh\"" Jul 28 17:10:52.758518 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.758477 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cpt5q\"" Jul 28 17:10:52.766183 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.766138 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9m9d6"] Jul 28 17:10:52.773075 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.773055 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qqpxl"] Jul 28 17:10:52.880651 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.880624 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-bound-sa-token\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.880840 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.880684 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/22ace845-d4cb-4788-b099-8b4990ca80e8-image-registry-private-configuration\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.880840 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.880709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-certificates\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.880840 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.880733 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0218446-ab4e-4002-9f70-1d5c5c222bed-cert\") pod \"ingress-canary-qqpxl\" (UID: \"c0218446-ab4e-4002-9f70-1d5c5c222bed\") " pod="openshift-ingress-canary/ingress-canary-qqpxl" Jul 28 17:10:52.880840 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.880757 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34c245a9-ee7b-4561-aebf-43c4c325c332-config-volume\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:10:52.880840 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.880778 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th9hk\" (UniqueName: \"kubernetes.io/projected/c0218446-ab4e-4002-9f70-1d5c5c222bed-kube-api-access-th9hk\") pod \"ingress-canary-qqpxl\" (UID: \"c0218446-ab4e-4002-9f70-1d5c5c222bed\") " pod="openshift-ingress-canary/ingress-canary-qqpxl" Jul 28 17:10:52.881125 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.880849 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.881125 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.880906 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22ace845-d4cb-4788-b099-8b4990ca80e8-trusted-ca\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.881125 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.880949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:10:52.881125 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.881005 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxlrq\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-kube-api-access-cxlrq\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.881125 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.881040 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqvqh\" (UniqueName: \"kubernetes.io/projected/34c245a9-ee7b-4561-aebf-43c4c325c332-kube-api-access-vqvqh\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:10:52.881125 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.881071 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/22ace845-d4cb-4788-b099-8b4990ca80e8-ca-trust-extracted\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.881350 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.881130 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/22ace845-d4cb-4788-b099-8b4990ca80e8-installation-pull-secrets\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.881350 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.881157 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34c245a9-ee7b-4561-aebf-43c4c325c332-tmp-dir\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:10:52.981701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.981660 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/22ace845-d4cb-4788-b099-8b4990ca80e8-installation-pull-secrets\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.981701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.981706 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34c245a9-ee7b-4561-aebf-43c4c325c332-tmp-dir\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:10:52.981968 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.981733 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-bound-sa-token\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.981968 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.981783 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/22ace845-d4cb-4788-b099-8b4990ca80e8-image-registry-private-configuration\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.981968 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.981818 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-certificates\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.981968 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.981838 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0218446-ab4e-4002-9f70-1d5c5c222bed-cert\") pod \"ingress-canary-qqpxl\" (UID: \"c0218446-ab4e-4002-9f70-1d5c5c222bed\") " pod="openshift-ingress-canary/ingress-canary-qqpxl" Jul 28 17:10:52.981968 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.981858 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34c245a9-ee7b-4561-aebf-43c4c325c332-config-volume\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:10:52.981968 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.981892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th9hk\" (UniqueName: \"kubernetes.io/projected/c0218446-ab4e-4002-9f70-1d5c5c222bed-kube-api-access-th9hk\") pod \"ingress-canary-qqpxl\" (UID: \"c0218446-ab4e-4002-9f70-1d5c5c222bed\") " pod="openshift-ingress-canary/ingress-canary-qqpxl" Jul 28 17:10:52.981968 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.981930 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.981968 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.981958 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22ace845-d4cb-4788-b099-8b4990ca80e8-trusted-ca\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.982337 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.981986 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:10:52.982337 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.982018 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxlrq\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-kube-api-access-cxlrq\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.982337 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.982042 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqvqh\" (UniqueName: \"kubernetes.io/projected/34c245a9-ee7b-4561-aebf-43c4c325c332-kube-api-access-vqvqh\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:10:52.982337 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.982065 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/22ace845-d4cb-4788-b099-8b4990ca80e8-ca-trust-extracted\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.982337 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.982094 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34c245a9-ee7b-4561-aebf-43c4c325c332-tmp-dir\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:10:52.982558 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.982446 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/22ace845-d4cb-4788-b099-8b4990ca80e8-ca-trust-extracted\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.982558 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:52.982551 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:10:52.982661 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:52.982564 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b54dc8599-qn4bf: secret "image-registry-tls" not found Jul 28 17:10:52.982661 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:52.982613 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls podName:22ace845-d4cb-4788-b099-8b4990ca80e8 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:53.482596489 +0000 UTC m=+36.083048822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls") pod "image-registry-7b54dc8599-qn4bf" (UID: "22ace845-d4cb-4788-b099-8b4990ca80e8") : secret "image-registry-tls" not found Jul 28 17:10:52.982867 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.982846 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-certificates\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.982999 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.982981 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34c245a9-ee7b-4561-aebf-43c4c325c332-config-volume\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:10:52.983406 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:52.983384 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:10:52.983501 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:52.983454 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls podName:34c245a9-ee7b-4561-aebf-43c4c325c332 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:53.48343746 +0000 UTC m=+36.083889802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls") pod "dns-default-9m9d6" (UID: "34c245a9-ee7b-4561-aebf-43c4c325c332") : secret "dns-default-metrics-tls" not found Jul 28 17:10:52.983615 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.983556 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22ace845-d4cb-4788-b099-8b4990ca80e8-trusted-ca\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.986685 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.986660 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/22ace845-d4cb-4788-b099-8b4990ca80e8-installation-pull-secrets\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.986797 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.986719 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/22ace845-d4cb-4788-b099-8b4990ca80e8-image-registry-private-configuration\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.987062 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.987037 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0218446-ab4e-4002-9f70-1d5c5c222bed-cert\") pod \"ingress-canary-qqpxl\" (UID: \"c0218446-ab4e-4002-9f70-1d5c5c222bed\") " pod="openshift-ingress-canary/ingress-canary-qqpxl" Jul 28 17:10:52.993564 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.993532 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxlrq\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-kube-api-access-cxlrq\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.993768 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.993728 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th9hk\" (UniqueName: \"kubernetes.io/projected/c0218446-ab4e-4002-9f70-1d5c5c222bed-kube-api-access-th9hk\") pod \"ingress-canary-qqpxl\" (UID: \"c0218446-ab4e-4002-9f70-1d5c5c222bed\") " pod="openshift-ingress-canary/ingress-canary-qqpxl" Jul 28 17:10:52.993853 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.993825 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-bound-sa-token\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:52.995442 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:52.995422 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqvqh\" (UniqueName: \"kubernetes.io/projected/34c245a9-ee7b-4561-aebf-43c4c325c332-kube-api-access-vqvqh\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:10:53.063555 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:53.063517 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qqpxl" Jul 28 17:10:53.206571 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:53.206294 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qqpxl"] Jul 28 17:10:53.208703 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:10:53.208672 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0218446_ab4e_4002_9f70_1d5c5c222bed.slice/crio-226b959b782572ff4cc4e097e267c508abf1a6c5f7c5ddf587303e112e73a9e1 WatchSource:0}: Error finding container 226b959b782572ff4cc4e097e267c508abf1a6c5f7c5ddf587303e112e73a9e1: Status 404 returned error can't find the container with id 226b959b782572ff4cc4e097e267c508abf1a6c5f7c5ddf587303e112e73a9e1 Jul 28 17:10:53.486571 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:53.486529 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:10:53.486768 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:53.486642 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:53.486768 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:53.486690 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:10:53.486768 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:53.486758 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:10:53.486768 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:53.486765 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls podName:34c245a9-ee7b-4561-aebf-43c4c325c332 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:54.486749217 +0000 UTC m=+37.087201541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls") pod "dns-default-9m9d6" (UID: "34c245a9-ee7b-4561-aebf-43c4c325c332") : secret "dns-default-metrics-tls" not found Jul 28 17:10:53.486768 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:53.486771 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b54dc8599-qn4bf: secret "image-registry-tls" not found Jul 28 17:10:53.487008 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:53.486821 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls podName:22ace845-d4cb-4788-b099-8b4990ca80e8 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:54.486805664 +0000 UTC m=+37.087257990 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls") pod "image-registry-7b54dc8599-qn4bf" (UID: "22ace845-d4cb-4788-b099-8b4990ca80e8") : secret "image-registry-tls" not found Jul 28 17:10:53.944734 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:53.944701 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:10:53.944734 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:53.944735 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:10:53.945535 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:53.944901 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:10:53.949212 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:53.949185 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 17:10:53.950821 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:53.950408 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 17:10:53.950821 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:53.950445 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 17:10:53.950821 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:53.950415 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wz9mb\"" Jul 28 17:10:53.950821 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:53.950463 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 17:10:53.950821 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:53.950573 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xf5sm\"" Jul 28 17:10:54.090267 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:54.090227 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qqpxl" event={"ID":"c0218446-ab4e-4002-9f70-1d5c5c222bed","Type":"ContainerStarted","Data":"226b959b782572ff4cc4e097e267c508abf1a6c5f7c5ddf587303e112e73a9e1"} Jul 28 17:10:54.093511 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:54.093474 2562 generic.go:356] "Generic (PLEG): container finished" podID="3f00b9d9-ad3e-4e58-8faa-da2eed76d38e" containerID="5da0014a729d0ad2714cf36fe62f9b8cddee7746baca4ae0a205c38ca53c20c5" exitCode=0 Jul 28 17:10:54.093649 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:54.093526 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs8m6" event={"ID":"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e","Type":"ContainerDied","Data":"5da0014a729d0ad2714cf36fe62f9b8cddee7746baca4ae0a205c38ca53c20c5"} Jul 28 17:10:54.495683 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:54.495645 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:54.495868 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:54.495704 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:10:54.495868 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:54.495822 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:10:54.495868 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:54.495826 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:10:54.496047 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:54.495917 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls podName:34c245a9-ee7b-4561-aebf-43c4c325c332 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:56.495896474 +0000 UTC m=+39.096348817 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls") pod "dns-default-9m9d6" (UID: "34c245a9-ee7b-4561-aebf-43c4c325c332") : secret "dns-default-metrics-tls" not found Jul 28 17:10:54.496047 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:54.495841 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b54dc8599-qn4bf: secret "image-registry-tls" not found Jul 28 17:10:54.496047 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:54.496033 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls podName:22ace845-d4cb-4788-b099-8b4990ca80e8 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:56.496008998 +0000 UTC m=+39.096461347 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls") pod "image-registry-7b54dc8599-qn4bf" (UID: "22ace845-d4cb-4788-b099-8b4990ca80e8") : secret "image-registry-tls" not found Jul 28 17:10:55.099491 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:55.099456 2562 generic.go:356] "Generic (PLEG): container finished" podID="3f00b9d9-ad3e-4e58-8faa-da2eed76d38e" containerID="f0670864bd25f6752fb2cbfb6311b5a9eda22af71212640fc2053d1523ee3f19" exitCode=0 Jul 28 17:10:55.100010 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:55.099519 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs8m6" event={"ID":"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e","Type":"ContainerDied","Data":"f0670864bd25f6752fb2cbfb6311b5a9eda22af71212640fc2053d1523ee3f19"} Jul 28 17:10:56.103416 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:56.103207 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qqpxl" event={"ID":"c0218446-ab4e-4002-9f70-1d5c5c222bed","Type":"ContainerStarted","Data":"74eab2c8964f7ac068799301a2453538120d5cf1a0366435e6b21813abf5dc4b"} Jul 28 17:10:56.106799 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:56.106774 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs8m6" event={"ID":"3f00b9d9-ad3e-4e58-8faa-da2eed76d38e","Type":"ContainerStarted","Data":"e56d8acceb2f684d2835207b4841bc70601056f4b716c02ec5bfea091b506dc0"} Jul 28 17:10:56.121696 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:56.121621 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qqpxl" podStartSLOduration=1.6520834290000002 podStartE2EDuration="4.121606471s" totalImagesPullingTime="2.469523042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:56.121450705 +0000 UTC m=+38.721903051" watchObservedRunningTime="2026-07-28 17:10:56.121606471 +0000 UTC m=+38.722058827" Jul 28 17:10:56.147630 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:56.147575 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zs8m6" podStartSLOduration=9.510123918 podStartE2EDuration="38.147555716s" totalImagesPullingTime="28.637431798s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:56.145770392 +0000 UTC m=+38.746222737" watchObservedRunningTime="2026-07-28 17:10:56.147555716 +0000 UTC m=+38.748008131" Jul 28 17:10:56.513531 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:56.513491 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:10:56.513531 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:10:56.513540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:10:56.513755 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:56.513634 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:10:56.513755 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:56.513641 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:10:56.513755 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:56.513663 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b54dc8599-qn4bf: secret "image-registry-tls" not found Jul 28 17:10:56.513755 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:56.513688 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls podName:34c245a9-ee7b-4561-aebf-43c4c325c332 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:00.51367549 +0000 UTC m=+43.114127813 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls") pod "dns-default-9m9d6" (UID: "34c245a9-ee7b-4561-aebf-43c4c325c332") : secret "dns-default-metrics-tls" not found Jul 28 17:10:56.513755 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:10:56.513712 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls podName:22ace845-d4cb-4788-b099-8b4990ca80e8 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:00.513694212 +0000 UTC m=+43.114146536 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls") pod "image-registry-7b54dc8599-qn4bf" (UID: "22ace845-d4cb-4788-b099-8b4990ca80e8") : secret "image-registry-tls" not found Jul 28 17:11:00.539678 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:00.539634 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:11:00.539678 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:00.539686 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:11:00.540122 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:00.539783 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:11:00.540122 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:00.539801 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b54dc8599-qn4bf: secret "image-registry-tls" not found Jul 28 17:11:00.540122 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:00.539807 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:11:00.540122 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:00.539856 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls podName:34c245a9-ee7b-4561-aebf-43c4c325c332 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:08.539842102 +0000 UTC m=+51.140294426 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls") pod "dns-default-9m9d6" (UID: "34c245a9-ee7b-4561-aebf-43c4c325c332") : secret "dns-default-metrics-tls" not found Jul 28 17:11:00.540122 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:00.539870 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls podName:22ace845-d4cb-4788-b099-8b4990ca80e8 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:08.539863941 +0000 UTC m=+51.140316264 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls") pod "image-registry-7b54dc8599-qn4bf" (UID: "22ace845-d4cb-4788-b099-8b4990ca80e8") : secret "image-registry-tls" not found Jul 28 17:11:00.943646 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:00.943573 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-lgwb7"] Jul 28 17:11:00.973219 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:00.973186 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-lgwb7"] Jul 28 17:11:00.973367 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:00.973294 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-lgwb7" Jul 28 17:11:00.976611 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:00.976588 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 28 17:11:00.977887 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:00.977844 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 28 17:11:00.978020 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:00.977844 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 28 17:11:00.978020 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:00.977905 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-r5jx8\"" Jul 28 17:11:00.978020 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:00.977899 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 28 17:11:01.044616 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:01.044566 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/210d0929-e2cf-4854-a112-cbbe57ffbd19-signing-key\") pod \"service-ca-658d7b7c78-lgwb7\" (UID: \"210d0929-e2cf-4854-a112-cbbe57ffbd19\") " pod="openshift-service-ca/service-ca-658d7b7c78-lgwb7" Jul 28 17:11:01.044792 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:01.044639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/210d0929-e2cf-4854-a112-cbbe57ffbd19-signing-cabundle\") pod \"service-ca-658d7b7c78-lgwb7\" (UID: \"210d0929-e2cf-4854-a112-cbbe57ffbd19\") " pod="openshift-service-ca/service-ca-658d7b7c78-lgwb7" Jul 28 17:11:01.044792 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:01.044713 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcrxz\" (UniqueName: \"kubernetes.io/projected/210d0929-e2cf-4854-a112-cbbe57ffbd19-kube-api-access-gcrxz\") pod \"service-ca-658d7b7c78-lgwb7\" (UID: \"210d0929-e2cf-4854-a112-cbbe57ffbd19\") " pod="openshift-service-ca/service-ca-658d7b7c78-lgwb7" Jul 28 17:11:01.146097 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:01.146054 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/210d0929-e2cf-4854-a112-cbbe57ffbd19-signing-key\") pod \"service-ca-658d7b7c78-lgwb7\" (UID: \"210d0929-e2cf-4854-a112-cbbe57ffbd19\") " pod="openshift-service-ca/service-ca-658d7b7c78-lgwb7" Jul 28 17:11:01.146097 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:01.146093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/210d0929-e2cf-4854-a112-cbbe57ffbd19-signing-cabundle\") pod \"service-ca-658d7b7c78-lgwb7\" (UID: \"210d0929-e2cf-4854-a112-cbbe57ffbd19\") " pod="openshift-service-ca/service-ca-658d7b7c78-lgwb7" Jul 28 17:11:01.146322 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:01.146133 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcrxz\" (UniqueName: \"kubernetes.io/projected/210d0929-e2cf-4854-a112-cbbe57ffbd19-kube-api-access-gcrxz\") pod \"service-ca-658d7b7c78-lgwb7\" (UID: \"210d0929-e2cf-4854-a112-cbbe57ffbd19\") " pod="openshift-service-ca/service-ca-658d7b7c78-lgwb7" Jul 28 17:11:01.146900 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:01.146855 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/210d0929-e2cf-4854-a112-cbbe57ffbd19-signing-cabundle\") pod \"service-ca-658d7b7c78-lgwb7\" (UID: \"210d0929-e2cf-4854-a112-cbbe57ffbd19\") " pod="openshift-service-ca/service-ca-658d7b7c78-lgwb7" Jul 28 17:11:01.150344 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:01.150319 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/210d0929-e2cf-4854-a112-cbbe57ffbd19-signing-key\") pod \"service-ca-658d7b7c78-lgwb7\" (UID: \"210d0929-e2cf-4854-a112-cbbe57ffbd19\") " pod="openshift-service-ca/service-ca-658d7b7c78-lgwb7" Jul 28 17:11:01.155000 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:01.154978 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcrxz\" (UniqueName: \"kubernetes.io/projected/210d0929-e2cf-4854-a112-cbbe57ffbd19-kube-api-access-gcrxz\") pod \"service-ca-658d7b7c78-lgwb7\" (UID: \"210d0929-e2cf-4854-a112-cbbe57ffbd19\") " pod="openshift-service-ca/service-ca-658d7b7c78-lgwb7" Jul 28 17:11:01.282723 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:01.282685 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-lgwb7" Jul 28 17:11:01.422213 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:01.422182 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-lgwb7"] Jul 28 17:11:01.424920 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:01.424891 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod210d0929_e2cf_4854_a112_cbbe57ffbd19.slice/crio-b8abdd7ff460d85a7b9441f54c141c8cf09b8c742ce014b56cd009f77c5fc289 WatchSource:0}: Error finding container b8abdd7ff460d85a7b9441f54c141c8cf09b8c742ce014b56cd009f77c5fc289: Status 404 returned error can't find the container with id b8abdd7ff460d85a7b9441f54c141c8cf09b8c742ce014b56cd009f77c5fc289 Jul 28 17:11:02.121290 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:02.121250 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-lgwb7" event={"ID":"210d0929-e2cf-4854-a112-cbbe57ffbd19","Type":"ContainerStarted","Data":"b8abdd7ff460d85a7b9441f54c141c8cf09b8c742ce014b56cd009f77c5fc289"} Jul 28 17:11:04.128004 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:04.127962 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-lgwb7" event={"ID":"210d0929-e2cf-4854-a112-cbbe57ffbd19","Type":"ContainerStarted","Data":"88357aeaa3c2cb4b3c4153556c7867889787844b815710e9e0dd45bec82aaacc"} Jul 28 17:11:04.146009 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:04.145787 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-lgwb7" podStartSLOduration=1.615667782 podStartE2EDuration="4.145766852s" totalImagesPullingTime="2.53009907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:04.145517961 +0000 UTC m=+46.745970321" watchObservedRunningTime="2026-07-28 17:11:04.145766852 +0000 UTC m=+46.746219197" Jul 28 17:11:08.602363 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:08.602323 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:11:08.602785 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:08.602384 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:11:08.604569 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:08.604541 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34c245a9-ee7b-4561-aebf-43c4c325c332-metrics-tls\") pod \"dns-default-9m9d6\" (UID: \"34c245a9-ee7b-4561-aebf-43c4c325c332\") " pod="openshift-dns/dns-default-9m9d6" Jul 28 17:11:08.604739 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:08.604720 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls\") pod \"image-registry-7b54dc8599-qn4bf\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:11:08.634698 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:08.634659 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:11:08.671178 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:08.671155 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9m9d6" Jul 28 17:11:08.823319 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:08.823289 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b54dc8599-qn4bf"] Jul 28 17:11:08.826660 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:08.826635 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ace845_d4cb_4788_b099_8b4990ca80e8.slice/crio-54dba4b288e1038af5cd817bbd5bc47f356b9c060e449d2b5f8784b0a634d917 WatchSource:0}: Error finding container 54dba4b288e1038af5cd817bbd5bc47f356b9c060e449d2b5f8784b0a634d917: Status 404 returned error can't find the container with id 54dba4b288e1038af5cd817bbd5bc47f356b9c060e449d2b5f8784b0a634d917 Jul 28 17:11:08.837985 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:08.837958 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9m9d6"] Jul 28 17:11:08.841012 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:08.840986 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34c245a9_ee7b_4561_aebf_43c4c325c332.slice/crio-e58ffdc7b5523229fc9f4a8e61d8e1a7fb3da9a0fe43182d6def18e9830e7feb WatchSource:0}: Error finding container e58ffdc7b5523229fc9f4a8e61d8e1a7fb3da9a0fe43182d6def18e9830e7feb: Status 404 returned error can't find the container with id e58ffdc7b5523229fc9f4a8e61d8e1a7fb3da9a0fe43182d6def18e9830e7feb Jul 28 17:11:09.144954 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:09.141637 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9m9d6" event={"ID":"34c245a9-ee7b-4561-aebf-43c4c325c332","Type":"ContainerStarted","Data":"e58ffdc7b5523229fc9f4a8e61d8e1a7fb3da9a0fe43182d6def18e9830e7feb"} Jul 28 17:11:09.144954 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:09.143307 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" event={"ID":"22ace845-d4cb-4788-b099-8b4990ca80e8","Type":"ContainerStarted","Data":"114f1c1e2a0ecccdf7430a4decf67d741c6092f4e8aefe4c0a598af40aa035ef"} Jul 28 17:11:09.144954 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:09.143349 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" event={"ID":"22ace845-d4cb-4788-b099-8b4990ca80e8","Type":"ContainerStarted","Data":"54dba4b288e1038af5cd817bbd5bc47f356b9c060e449d2b5f8784b0a634d917"} Jul 28 17:11:09.144954 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:09.143451 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:11:09.166343 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:09.166276 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" podStartSLOduration=53.166253807 podStartE2EDuration="53.166253807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:09.165278517 +0000 UTC m=+51.765730875" watchObservedRunningTime="2026-07-28 17:11:09.166253807 +0000 UTC m=+51.766706154" Jul 28 17:11:12.152967 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:12.152934 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9m9d6" event={"ID":"34c245a9-ee7b-4561-aebf-43c4c325c332","Type":"ContainerStarted","Data":"7ad87a502d6ff46b4ac320c3c831350d3282c34cc93e6fd63c2b8999a47304f8"} Jul 28 17:11:12.153394 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:12.152990 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9m9d6" event={"ID":"34c245a9-ee7b-4561-aebf-43c4c325c332","Type":"ContainerStarted","Data":"635f6444ab075bdebb4a564d8d91e34d68f9964981d50d8fd735af07346a9de4"} Jul 28 17:11:12.153394 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:12.153025 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9m9d6" Jul 28 17:11:12.172951 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:12.172903 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9m9d6" podStartSLOduration=17.941817077 podStartE2EDuration="20.172887325s" totalImagesPullingTime="2.231070248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:12.171615435 +0000 UTC m=+54.772067781" watchObservedRunningTime="2026-07-28 17:11:12.172887325 +0000 UTC m=+54.773339663" Jul 28 17:11:18.091020 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:18.090992 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kxpf9" Jul 28 17:11:21.465023 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.464983 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-q5w5v"] Jul 28 17:11:21.468453 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.468426 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.472248 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.472225 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 28 17:11:21.472507 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.472492 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 28 17:11:21.473707 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.473690 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-jtnw5\"" Jul 28 17:11:21.473805 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.473690 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 28 17:11:21.480422 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.480401 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 28 17:11:21.492052 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.492024 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 28 17:11:21.496685 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.496662 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-q5w5v"] Jul 28 17:11:21.504553 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.504524 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a72944f0-79e5-407d-9add-0c510b4bc4a8-service-ca-bundle\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.504654 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.504563 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a72944f0-79e5-407d-9add-0c510b4bc4a8-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.504654 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.504585 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a72944f0-79e5-407d-9add-0c510b4bc4a8-tmp\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.504654 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.504630 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwqdr\" (UniqueName: \"kubernetes.io/projected/a72944f0-79e5-407d-9add-0c510b4bc4a8-kube-api-access-bwqdr\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.504770 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.504674 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a72944f0-79e5-407d-9add-0c510b4bc4a8-snapshots\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.504770 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.504690 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a72944f0-79e5-407d-9add-0c510b4bc4a8-serving-cert\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.605784 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.605742 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a72944f0-79e5-407d-9add-0c510b4bc4a8-snapshots\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.605784 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.605782 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a72944f0-79e5-407d-9add-0c510b4bc4a8-serving-cert\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.606064 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.605805 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a72944f0-79e5-407d-9add-0c510b4bc4a8-service-ca-bundle\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.606064 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.605826 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a72944f0-79e5-407d-9add-0c510b4bc4a8-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.606064 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.605960 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a72944f0-79e5-407d-9add-0c510b4bc4a8-tmp\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.606064 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.606023 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwqdr\" (UniqueName: \"kubernetes.io/projected/a72944f0-79e5-407d-9add-0c510b4bc4a8-kube-api-access-bwqdr\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.606495 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.606457 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a72944f0-79e5-407d-9add-0c510b4bc4a8-tmp\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.606623 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.606561 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a72944f0-79e5-407d-9add-0c510b4bc4a8-snapshots\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.606696 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.606677 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a72944f0-79e5-407d-9add-0c510b4bc4a8-service-ca-bundle\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.606763 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.606747 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a72944f0-79e5-407d-9add-0c510b4bc4a8-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.608304 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.608283 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a72944f0-79e5-407d-9add-0c510b4bc4a8-serving-cert\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.609033 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.609008 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m96t9"] Jul 28 17:11:21.611034 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.611017 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m96t9" Jul 28 17:11:21.619300 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.619281 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5d944fd6b9-xmtww"] Jul 28 17:11:21.621211 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.621196 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:21.629046 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:21.629022 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-metrics-certs-default\" is forbidden: User \"system:node:ip-10-0-139-193.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-139-193.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" type="*v1.Secret" Jul 28 17:11:21.629122 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:21.629053 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-dockercfg-jdj8v\" is forbidden: User \"system:node:ip-10-0-139-193.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-139-193.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jdj8v\"" type="*v1.Secret" Jul 28 17:11:21.632701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.632686 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:11:21.632767 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:21.632694 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-ingress-cert\" is forbidden: User \"system:node:ip-10-0-139-193.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-139-193.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" type="*v1.Secret" Jul 28 17:11:21.632767 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:21.632705 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-139-193.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-139-193.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 28 17:11:21.632943 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.632929 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-jndrr\"" Jul 28 17:11:21.633491 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.633477 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 28 17:11:21.633972 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:21.633953 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-139-193.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-139-193.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 28 17:11:21.634009 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:21.633975 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-stats-default\" is forbidden: User \"system:node:ip-10-0-139-193.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-139-193.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" type="*v1.Secret" Jul 28 17:11:21.635050 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.635037 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 28 17:11:21.652523 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.652495 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-d7xlp"] Jul 28 17:11:21.654322 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.654303 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:21.655490 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.655470 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwqdr\" (UniqueName: \"kubernetes.io/projected/a72944f0-79e5-407d-9add-0c510b4bc4a8-kube-api-access-bwqdr\") pod \"insights-operator-5885fdb759-q5w5v\" (UID: \"a72944f0-79e5-407d-9add-0c510b4bc4a8\") " pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.663073 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.663054 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 28 17:11:21.663167 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.663076 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-rdwzm\"" Jul 28 17:11:21.663649 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.663632 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 28 17:11:21.663943 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.663849 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:11:21.664671 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.664655 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 28 17:11:21.673268 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.673245 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d944fd6b9-xmtww"] Jul 28 17:11:21.673549 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.673531 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 28 17:11:21.690560 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.690536 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m96t9"] Jul 28 17:11:21.706665 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.706639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d779727-dc91-4c3d-863a-622cbfdfbd47-metrics-certs\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:21.706802 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.706671 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baa763a5-35b5-4ab4-8a81-54c23b7de388-trusted-ca\") pod \"console-operator-584dcb6b54-d7xlp\" (UID: \"baa763a5-35b5-4ab4-8a81-54c23b7de388\") " pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:21.706802 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.706694 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5d779727-dc91-4c3d-863a-622cbfdfbd47-stats-auth\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:21.706802 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.706750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d779727-dc91-4c3d-863a-622cbfdfbd47-service-ca-bundle\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:21.706802 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.706770 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsgdx\" (UniqueName: \"kubernetes.io/projected/5d779727-dc91-4c3d-863a-622cbfdfbd47-kube-api-access-lsgdx\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:21.706802 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.706800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbc4w\" (UniqueName: \"kubernetes.io/projected/baa763a5-35b5-4ab4-8a81-54c23b7de388-kube-api-access-cbc4w\") pod \"console-operator-584dcb6b54-d7xlp\" (UID: \"baa763a5-35b5-4ab4-8a81-54c23b7de388\") " pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:21.707103 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.706858 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/baa763a5-35b5-4ab4-8a81-54c23b7de388-config\") pod \"console-operator-584dcb6b54-d7xlp\" (UID: \"baa763a5-35b5-4ab4-8a81-54c23b7de388\") " pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:21.707103 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.706906 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5d779727-dc91-4c3d-863a-622cbfdfbd47-default-certificate\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:21.707103 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.706925 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/baa763a5-35b5-4ab4-8a81-54c23b7de388-serving-cert\") pod \"console-operator-584dcb6b54-d7xlp\" (UID: \"baa763a5-35b5-4ab4-8a81-54c23b7de388\") " pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:21.707103 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.706949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgv9f\" (UniqueName: \"kubernetes.io/projected/c4dae54a-3fc7-4b5b-aa8e-c08155e12eca-kube-api-access-qgv9f\") pod \"volume-data-source-validator-549c6d54cf-m96t9\" (UID: \"c4dae54a-3fc7-4b5b-aa8e-c08155e12eca\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m96t9" Jul 28 17:11:21.721788 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.720350 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-d7xlp"] Jul 28 17:11:21.776893 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.776850 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-q5w5v" Jul 28 17:11:21.807457 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.807415 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d779727-dc91-4c3d-863a-622cbfdfbd47-metrics-certs\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:21.807457 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.807457 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baa763a5-35b5-4ab4-8a81-54c23b7de388-trusted-ca\") pod \"console-operator-584dcb6b54-d7xlp\" (UID: \"baa763a5-35b5-4ab4-8a81-54c23b7de388\") " pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:21.807709 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.807493 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5d779727-dc91-4c3d-863a-622cbfdfbd47-stats-auth\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:21.807709 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.807570 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d779727-dc91-4c3d-863a-622cbfdfbd47-service-ca-bundle\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:21.807709 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.807601 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsgdx\" (UniqueName: \"kubernetes.io/projected/5d779727-dc91-4c3d-863a-622cbfdfbd47-kube-api-access-lsgdx\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:21.807709 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.807635 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbc4w\" (UniqueName: \"kubernetes.io/projected/baa763a5-35b5-4ab4-8a81-54c23b7de388-kube-api-access-cbc4w\") pod \"console-operator-584dcb6b54-d7xlp\" (UID: \"baa763a5-35b5-4ab4-8a81-54c23b7de388\") " pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:21.807709 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.807679 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/baa763a5-35b5-4ab4-8a81-54c23b7de388-config\") pod \"console-operator-584dcb6b54-d7xlp\" (UID: \"baa763a5-35b5-4ab4-8a81-54c23b7de388\") " pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:21.807709 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.807708 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5d779727-dc91-4c3d-863a-622cbfdfbd47-default-certificate\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:21.808026 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.807727 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/baa763a5-35b5-4ab4-8a81-54c23b7de388-serving-cert\") pod \"console-operator-584dcb6b54-d7xlp\" (UID: \"baa763a5-35b5-4ab4-8a81-54c23b7de388\") " pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:21.808026 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.807762 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgv9f\" (UniqueName: \"kubernetes.io/projected/c4dae54a-3fc7-4b5b-aa8e-c08155e12eca-kube-api-access-qgv9f\") pod \"volume-data-source-validator-549c6d54cf-m96t9\" (UID: \"c4dae54a-3fc7-4b5b-aa8e-c08155e12eca\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m96t9" Jul 28 17:11:21.808469 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.808445 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baa763a5-35b5-4ab4-8a81-54c23b7de388-trusted-ca\") pod \"console-operator-584dcb6b54-d7xlp\" (UID: \"baa763a5-35b5-4ab4-8a81-54c23b7de388\") " pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:21.808544 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.808454 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d779727-dc91-4c3d-863a-622cbfdfbd47-service-ca-bundle\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:21.808544 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.808522 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/baa763a5-35b5-4ab4-8a81-54c23b7de388-config\") pod \"console-operator-584dcb6b54-d7xlp\" (UID: \"baa763a5-35b5-4ab4-8a81-54c23b7de388\") " pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:21.810323 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.810300 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/baa763a5-35b5-4ab4-8a81-54c23b7de388-serving-cert\") pod \"console-operator-584dcb6b54-d7xlp\" (UID: \"baa763a5-35b5-4ab4-8a81-54c23b7de388\") " pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:21.842343 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.842260 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbc4w\" (UniqueName: \"kubernetes.io/projected/baa763a5-35b5-4ab4-8a81-54c23b7de388-kube-api-access-cbc4w\") pod \"console-operator-584dcb6b54-d7xlp\" (UID: \"baa763a5-35b5-4ab4-8a81-54c23b7de388\") " pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:21.842852 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.842827 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgv9f\" (UniqueName: \"kubernetes.io/projected/c4dae54a-3fc7-4b5b-aa8e-c08155e12eca-kube-api-access-qgv9f\") pod \"volume-data-source-validator-549c6d54cf-m96t9\" (UID: \"c4dae54a-3fc7-4b5b-aa8e-c08155e12eca\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m96t9" Jul 28 17:11:21.920791 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.920756 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m96t9" Jul 28 17:11:21.922144 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.922121 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-q5w5v"] Jul 28 17:11:21.924057 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:21.924036 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda72944f0_79e5_407d_9add_0c510b4bc4a8.slice/crio-a1254f3fd2d64f71fbc0c110f68eedc2a1849f26ad1236a0945e50b2af6eb3bc WatchSource:0}: Error finding container a1254f3fd2d64f71fbc0c110f68eedc2a1849f26ad1236a0945e50b2af6eb3bc: Status 404 returned error can't find the container with id a1254f3fd2d64f71fbc0c110f68eedc2a1849f26ad1236a0945e50b2af6eb3bc Jul 28 17:11:21.967609 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:21.967321 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:22.067725 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:22.067675 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m96t9"] Jul 28 17:11:22.071831 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:22.071806 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4dae54a_3fc7_4b5b_aa8e_c08155e12eca.slice/crio-abfc90d5b5893cdee26571d44030944964989bd9b7963406c0b0c52497b18cf1 WatchSource:0}: Error finding container abfc90d5b5893cdee26571d44030944964989bd9b7963406c0b0c52497b18cf1: Status 404 returned error can't find the container with id abfc90d5b5893cdee26571d44030944964989bd9b7963406c0b0c52497b18cf1 Jul 28 17:11:22.120248 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:22.120217 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-d7xlp"] Jul 28 17:11:22.123934 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:22.123904 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaa763a5_35b5_4ab4_8a81_54c23b7de388.slice/crio-e5924f5e71bd15ff5c69288c422c74c0096e59d2857f2f2521b2c02b565bd9d2 WatchSource:0}: Error finding container e5924f5e71bd15ff5c69288c422c74c0096e59d2857f2f2521b2c02b565bd9d2: Status 404 returned error can't find the container with id e5924f5e71bd15ff5c69288c422c74c0096e59d2857f2f2521b2c02b565bd9d2 Jul 28 17:11:22.157926 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:22.157898 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9m9d6" Jul 28 17:11:22.177744 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:22.177706 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m96t9" event={"ID":"c4dae54a-3fc7-4b5b-aa8e-c08155e12eca","Type":"ContainerStarted","Data":"abfc90d5b5893cdee26571d44030944964989bd9b7963406c0b0c52497b18cf1"} Jul 28 17:11:22.178857 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:22.178832 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" event={"ID":"baa763a5-35b5-4ab4-8a81-54c23b7de388","Type":"ContainerStarted","Data":"e5924f5e71bd15ff5c69288c422c74c0096e59d2857f2f2521b2c02b565bd9d2"} Jul 28 17:11:22.179853 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:22.179832 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-q5w5v" event={"ID":"a72944f0-79e5-407d-9add-0c510b4bc4a8","Type":"ContainerStarted","Data":"a1254f3fd2d64f71fbc0c110f68eedc2a1849f26ad1236a0945e50b2af6eb3bc"} Jul 28 17:11:22.482218 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:22.482185 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 28 17:11:22.492436 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:22.492405 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d779727-dc91-4c3d-863a-622cbfdfbd47-metrics-certs\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:22.503413 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:22.503388 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 28 17:11:22.621591 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:22.621554 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 28 17:11:22.625726 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:22.625698 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsgdx\" (UniqueName: \"kubernetes.io/projected/5d779727-dc91-4c3d-863a-622cbfdfbd47-kube-api-access-lsgdx\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:22.718868 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:22.718838 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jdj8v\"" Jul 28 17:11:22.799754 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:22.799667 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 28 17:11:22.808466 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:22.808442 2562 secret.go:189] Couldn't get secret openshift-ingress/router-stats-default: failed to sync secret cache: timed out waiting for the condition Jul 28 17:11:22.808622 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:22.808523 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d779727-dc91-4c3d-863a-622cbfdfbd47-stats-auth podName:5d779727-dc91-4c3d-863a-622cbfdfbd47 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:23.308503017 +0000 UTC m=+65.908955343 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "stats-auth" (UniqueName: "kubernetes.io/secret/5d779727-dc91-4c3d-863a-622cbfdfbd47-stats-auth") pod "router-default-5d944fd6b9-xmtww" (UID: "5d779727-dc91-4c3d-863a-622cbfdfbd47") : failed to sync secret cache: timed out waiting for the condition Jul 28 17:11:22.811661 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:22.811637 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5d779727-dc91-4c3d-863a-622cbfdfbd47-default-certificate\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:23.058345 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.058260 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 28 17:11:23.323803 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.322346 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5d779727-dc91-4c3d-863a-622cbfdfbd47-stats-auth\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:23.325351 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.325326 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5d779727-dc91-4c3d-863a-622cbfdfbd47-stats-auth\") pod \"router-default-5d944fd6b9-xmtww\" (UID: \"5d779727-dc91-4c3d-863a-622cbfdfbd47\") " pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:23.428759 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.428722 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:23.625299 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.625212 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:11:23.628546 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.628523 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 17:11:23.638432 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.638405 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f49cd3a-ee50-43ee-a297-ca95c775c799-original-pull-secret\") pod \"global-pull-secret-syncer-xn76m\" (UID: \"4f49cd3a-ee50-43ee-a297-ca95c775c799\") " pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:11:23.659382 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.659354 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn76m" Jul 28 17:11:23.725888 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.725841 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs\") pod \"network-metrics-daemon-nq44j\" (UID: \"64efaaa1-f45f-431e-a2d7-9b05a714971e\") " pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:11:23.726074 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.725941 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vtwn\" (UniqueName: \"kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn\") pod \"network-check-target-mpsgl\" (UID: \"1ca5eb66-7139-4339-82cb-e0c5cabb18f4\") " pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:11:23.729221 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.729193 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 17:11:23.729341 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.729229 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 17:11:23.739650 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.739620 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64efaaa1-f45f-431e-a2d7-9b05a714971e-metrics-certs\") pod \"network-metrics-daemon-nq44j\" (UID: \"64efaaa1-f45f-431e-a2d7-9b05a714971e\") " pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:11:23.739969 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.739946 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 17:11:23.750358 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.750317 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vtwn\" (UniqueName: \"kubernetes.io/projected/1ca5eb66-7139-4339-82cb-e0c5cabb18f4-kube-api-access-2vtwn\") pod \"network-check-target-mpsgl\" (UID: \"1ca5eb66-7139-4339-82cb-e0c5cabb18f4\") " pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:11:23.969967 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.969940 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wz9mb\"" Jul 28 17:11:23.974838 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.974818 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xf5sm\"" Jul 28 17:11:23.977088 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.977056 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:11:23.982742 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:23.982720 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nq44j" Jul 28 17:11:24.188639 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:24.188599 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m96t9" event={"ID":"c4dae54a-3fc7-4b5b-aa8e-c08155e12eca","Type":"ContainerStarted","Data":"cf327873a210b5144e9d5bebf17765287f9080f8bc6d0304c5a2ab7614a655c9"} Jul 28 17:11:24.206983 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:24.206923 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m96t9" podStartSLOduration=2.002016449 podStartE2EDuration="3.20690658s" totalImagesPullingTime="1.204890131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:24.205830134 +0000 UTC m=+66.806282492" watchObservedRunningTime="2026-07-28 17:11:24.20690658 +0000 UTC m=+66.807358925" Jul 28 17:11:24.548539 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:24.548381 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d944fd6b9-xmtww"] Jul 28 17:11:24.551118 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:24.551069 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d779727_dc91_4c3d_863a_622cbfdfbd47.slice/crio-3353286ce9f4a530107dda52bfd0cf877b8e531dd6fa3764ecd9818b002950d3 WatchSource:0}: Error finding container 3353286ce9f4a530107dda52bfd0cf877b8e531dd6fa3764ecd9818b002950d3: Status 404 returned error can't find the container with id 3353286ce9f4a530107dda52bfd0cf877b8e531dd6fa3764ecd9818b002950d3 Jul 28 17:11:24.769084 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:24.769050 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nq44j"] Jul 28 17:11:24.772587 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:24.772531 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64efaaa1_f45f_431e_a2d7_9b05a714971e.slice/crio-3f6b6f8f7e95fe6747a6d8b67db9ef0d2898185cd20a2eeb105c7ebf2be84de1 WatchSource:0}: Error finding container 3f6b6f8f7e95fe6747a6d8b67db9ef0d2898185cd20a2eeb105c7ebf2be84de1: Status 404 returned error can't find the container with id 3f6b6f8f7e95fe6747a6d8b67db9ef0d2898185cd20a2eeb105c7ebf2be84de1 Jul 28 17:11:24.773997 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:24.773972 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xn76m"] Jul 28 17:11:24.776597 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:24.776575 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mpsgl"] Jul 28 17:11:24.777540 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:24.777513 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f49cd3a_ee50_43ee_a297_ca95c775c799.slice/crio-c441dbf4e6bfe3885b61b0a0b0e8036fa8c4fae04ac986c9590a2df520ffbeaa WatchSource:0}: Error finding container c441dbf4e6bfe3885b61b0a0b0e8036fa8c4fae04ac986c9590a2df520ffbeaa: Status 404 returned error can't find the container with id c441dbf4e6bfe3885b61b0a0b0e8036fa8c4fae04ac986c9590a2df520ffbeaa Jul 28 17:11:24.782417 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:24.782393 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ca5eb66_7139_4339_82cb_e0c5cabb18f4.slice/crio-cec86f58a4d4871040abfa0bbc7b804132f95425ab3ae7697fa402dcc4655e49 WatchSource:0}: Error finding container cec86f58a4d4871040abfa0bbc7b804132f95425ab3ae7697fa402dcc4655e49: Status 404 returned error can't find the container with id cec86f58a4d4871040abfa0bbc7b804132f95425ab3ae7697fa402dcc4655e49 Jul 28 17:11:24.953585 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:24.953514 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-s7zwq"] Jul 28 17:11:24.961444 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:24.961416 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:24.964376 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:24.964353 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mtzhc\"" Jul 28 17:11:24.964517 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:24.964387 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 28 17:11:24.964517 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:24.964430 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 28 17:11:24.969636 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:24.969616 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s7zwq"] Jul 28 17:11:25.034526 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.034493 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:25.034526 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.034533 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec7b5a58-1942-455b-8195-c72e4c57e316-data-volume\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:25.034722 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.034580 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ct2d\" (UniqueName: \"kubernetes.io/projected/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-api-access-8ct2d\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:25.034722 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.034600 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ec7b5a58-1942-455b-8195-c72e4c57e316-crio-socket\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:25.034722 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.034625 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:25.135383 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.135347 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ec7b5a58-1942-455b-8195-c72e4c57e316-crio-socket\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:25.135569 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.135391 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:25.135569 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.135432 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:25.135569 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.135462 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec7b5a58-1942-455b-8195-c72e4c57e316-data-volume\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:25.135569 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:25.135547 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:11:25.135775 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.135578 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ec7b5a58-1942-455b-8195-c72e4c57e316-crio-socket\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:25.135775 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:25.135596 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:11:25.135775 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:25.135630 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm podName:ec7b5a58-1942-455b-8195-c72e4c57e316 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:25.63560897 +0000 UTC m=+68.236061307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s7zwq" (UID: "ec7b5a58-1942-455b-8195-c72e4c57e316") : configmap "kube-rbac-proxy" not found Jul 28 17:11:25.135775 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:25.135647 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls podName:ec7b5a58-1942-455b-8195-c72e4c57e316 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:25.635640633 +0000 UTC m=+68.236092958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s7zwq" (UID: "ec7b5a58-1942-455b-8195-c72e4c57e316") : secret "insights-runtime-extractor-tls" not found Jul 28 17:11:25.135775 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.135660 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ct2d\" (UniqueName: \"kubernetes.io/projected/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-api-access-8ct2d\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:25.136018 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.135871 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec7b5a58-1942-455b-8195-c72e4c57e316-data-volume\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:25.146559 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.146530 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ct2d\" (UniqueName: \"kubernetes.io/projected/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-api-access-8ct2d\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:25.194075 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.194035 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nq44j" event={"ID":"64efaaa1-f45f-431e-a2d7-9b05a714971e","Type":"ContainerStarted","Data":"3f6b6f8f7e95fe6747a6d8b67db9ef0d2898185cd20a2eeb105c7ebf2be84de1"} Jul 28 17:11:25.195261 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.195228 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xn76m" event={"ID":"4f49cd3a-ee50-43ee-a297-ca95c775c799","Type":"ContainerStarted","Data":"c441dbf4e6bfe3885b61b0a0b0e8036fa8c4fae04ac986c9590a2df520ffbeaa"} Jul 28 17:11:25.196792 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.196771 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d944fd6b9-xmtww" event={"ID":"5d779727-dc91-4c3d-863a-622cbfdfbd47","Type":"ContainerStarted","Data":"fc7909104fdad3963cd92d5c52f050a1c404fbfa4fed370d6bd8e2bc4adcb7d5"} Jul 28 17:11:25.196916 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.196807 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d944fd6b9-xmtww" event={"ID":"5d779727-dc91-4c3d-863a-622cbfdfbd47","Type":"ContainerStarted","Data":"3353286ce9f4a530107dda52bfd0cf877b8e531dd6fa3764ecd9818b002950d3"} Jul 28 17:11:25.198069 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.198047 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mpsgl" event={"ID":"1ca5eb66-7139-4339-82cb-e0c5cabb18f4","Type":"ContainerStarted","Data":"cec86f58a4d4871040abfa0bbc7b804132f95425ab3ae7697fa402dcc4655e49"} Jul 28 17:11:25.199566 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.199546 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" event={"ID":"baa763a5-35b5-4ab4-8a81-54c23b7de388","Type":"ContainerStarted","Data":"0f879caa5e502969633fcf783b8592b80d18b2d49915bd6ce7480652451c7fe5"} Jul 28 17:11:25.199798 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.199774 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:25.201199 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.201177 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-q5w5v" event={"ID":"a72944f0-79e5-407d-9add-0c510b4bc4a8","Type":"ContainerStarted","Data":"2697aa7e798537f2f42bf0ecc0ac41627f010325791054bfaba2f2f638a59d83"} Jul 28 17:11:25.219471 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.219427 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5d944fd6b9-xmtww" podStartSLOduration=4.219411673 podStartE2EDuration="4.219411673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:25.21930823 +0000 UTC m=+67.819760576" watchObservedRunningTime="2026-07-28 17:11:25.219411673 +0000 UTC m=+67.819863997" Jul 28 17:11:25.239397 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.239172 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-q5w5v" podStartSLOduration=1.792513601 podStartE2EDuration="4.23915629s" totalImagesPullingTime="2.446642689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:25.238568076 +0000 UTC m=+67.839020421" watchObservedRunningTime="2026-07-28 17:11:25.23915629 +0000 UTC m=+67.839608630" Jul 28 17:11:25.261497 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.261445 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" podStartSLOduration=2.007730302 podStartE2EDuration="4.261423164s" totalImagesPullingTime="2.253692862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:25.260247948 +0000 UTC m=+67.860700294" watchObservedRunningTime="2026-07-28 17:11:25.261423164 +0000 UTC m=+67.861875560" Jul 28 17:11:25.429483 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.429382 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:25.432547 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.432519 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:25.645920 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.645383 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:25.645920 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.645487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:25.645920 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:25.645545 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:11:25.645920 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:25.645621 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm podName:ec7b5a58-1942-455b-8195-c72e4c57e316 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:26.645600007 +0000 UTC m=+69.246052333 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s7zwq" (UID: "ec7b5a58-1942-455b-8195-c72e4c57e316") : configmap "kube-rbac-proxy" not found Jul 28 17:11:25.645920 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:25.645672 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:11:25.645920 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:25.645740 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls podName:ec7b5a58-1942-455b-8195-c72e4c57e316 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:26.64572439 +0000 UTC m=+69.246176718 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s7zwq" (UID: "ec7b5a58-1942-455b-8195-c72e4c57e316") : secret "insights-runtime-extractor-tls" not found Jul 28 17:11:25.907232 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.907150 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-d7xlp" Jul 28 17:11:25.966979 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:25.966949 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9m9d6_34c245a9-ee7b-4561-aebf-43c4c325c332/dns/0.log" Jul 28 17:11:26.099157 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.099106 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-hblt4"] Jul 28 17:11:26.115377 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.115298 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-hblt4" Jul 28 17:11:26.124537 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.118926 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-hblt4"] Jul 28 17:11:26.124537 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.120082 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 28 17:11:26.124537 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.120351 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 28 17:11:26.124537 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.122470 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-mj2vc\"" Jul 28 17:11:26.143537 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.143515 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9m9d6_34c245a9-ee7b-4561-aebf-43c4c325c332/kube-rbac-proxy/0.log" Jul 28 17:11:26.207021 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.206912 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:26.208732 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.208516 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5d944fd6b9-xmtww" Jul 28 17:11:26.252185 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.252145 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxx8b\" (UniqueName: \"kubernetes.io/projected/86e8f2ce-1d58-4acc-8d6e-fbb4bc8d208c-kube-api-access-lxx8b\") pod \"downloads-7c97dc4555-hblt4\" (UID: \"86e8f2ce-1d58-4acc-8d6e-fbb4bc8d208c\") " pod="openshift-console/downloads-7c97dc4555-hblt4" Jul 28 17:11:26.353496 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.353375 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxx8b\" (UniqueName: \"kubernetes.io/projected/86e8f2ce-1d58-4acc-8d6e-fbb4bc8d208c-kube-api-access-lxx8b\") pod \"downloads-7c97dc4555-hblt4\" (UID: \"86e8f2ce-1d58-4acc-8d6e-fbb4bc8d208c\") " pod="openshift-console/downloads-7c97dc4555-hblt4" Jul 28 17:11:26.366472 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.366439 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxx8b\" (UniqueName: \"kubernetes.io/projected/86e8f2ce-1d58-4acc-8d6e-fbb4bc8d208c-kube-api-access-lxx8b\") pod \"downloads-7c97dc4555-hblt4\" (UID: \"86e8f2ce-1d58-4acc-8d6e-fbb4bc8d208c\") " pod="openshift-console/downloads-7c97dc4555-hblt4" Jul 28 17:11:26.431514 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.431298 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-hblt4" Jul 28 17:11:26.627371 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.627279 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-hblt4"] Jul 28 17:11:26.656832 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.656292 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:26.656832 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.656382 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:26.656832 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:26.656459 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:11:26.656832 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:26.656520 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm podName:ec7b5a58-1942-455b-8195-c72e4c57e316 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:28.656502396 +0000 UTC m=+71.256954722 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s7zwq" (UID: "ec7b5a58-1942-455b-8195-c72e4c57e316") : configmap "kube-rbac-proxy" not found Jul 28 17:11:26.656832 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:26.656564 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:11:26.656832 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:26.656668 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls podName:ec7b5a58-1942-455b-8195-c72e4c57e316 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:28.656622547 +0000 UTC m=+71.257074872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s7zwq" (UID: "ec7b5a58-1942-455b-8195-c72e4c57e316") : secret "insights-runtime-extractor-tls" not found Jul 28 17:11:26.730665 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:26.730625 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86e8f2ce_1d58_4acc_8d6e_fbb4bc8d208c.slice/crio-a56766ddbecf3aab4840a7379c18ae4735eb052874348e33d325d2cd433b059c WatchSource:0}: Error finding container a56766ddbecf3aab4840a7379c18ae4735eb052874348e33d325d2cd433b059c: Status 404 returned error can't find the container with id a56766ddbecf3aab4840a7379c18ae4735eb052874348e33d325d2cd433b059c Jul 28 17:11:26.920992 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.920964 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t4kz8"] Jul 28 17:11:26.941469 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.941441 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-frb7f_249db02c-8b5f-42ba-922e-871de558c0b1/dns-node-resolver/0.log" Jul 28 17:11:26.943011 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.942983 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t4kz8"] Jul 28 17:11:26.943147 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.943081 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t4kz8" Jul 28 17:11:26.948657 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.947228 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vghtn\"" Jul 28 17:11:26.948657 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:26.947486 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 28 17:11:27.060139 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:27.060112 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2e004692-883b-4311-acec-06713b142f67-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-t4kz8\" (UID: \"2e004692-883b-4311-acec-06713b142f67\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t4kz8" Jul 28 17:11:27.162803 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:27.162233 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2e004692-883b-4311-acec-06713b142f67-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-t4kz8\" (UID: \"2e004692-883b-4311-acec-06713b142f67\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t4kz8" Jul 28 17:11:27.162803 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:27.162495 2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 28 17:11:27.162803 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:27.162565 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e004692-883b-4311-acec-06713b142f67-tls-certificates podName:2e004692-883b-4311-acec-06713b142f67 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:27.662546173 +0000 UTC m=+70.262998503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/2e004692-883b-4311-acec-06713b142f67-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-t4kz8" (UID: "2e004692-883b-4311-acec-06713b142f67") : secret "prometheus-operator-admission-webhook-tls" not found Jul 28 17:11:27.213110 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:27.213072 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nq44j" event={"ID":"64efaaa1-f45f-431e-a2d7-9b05a714971e","Type":"ContainerStarted","Data":"6571704154bf1c226b04dead9654f7ea24bb8bd5e37bffeb4823bc227e59e024"} Jul 28 17:11:27.213294 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:27.213135 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nq44j" event={"ID":"64efaaa1-f45f-431e-a2d7-9b05a714971e","Type":"ContainerStarted","Data":"660f625bb80e7f3b2ce06362724d3bab8206dc8c49c15b562e72de5d2cd2cd0b"} Jul 28 17:11:27.214642 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:27.214610 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-hblt4" event={"ID":"86e8f2ce-1d58-4acc-8d6e-fbb4bc8d208c","Type":"ContainerStarted","Data":"a56766ddbecf3aab4840a7379c18ae4735eb052874348e33d325d2cd433b059c"} Jul 28 17:11:27.240994 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:27.240937 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nq44j" podStartSLOduration=67.256915873 podStartE2EDuration="1m9.240911884s" totalImagesPullingTime="1.983996011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:27.238555041 +0000 UTC m=+69.839007387" watchObservedRunningTime="2026-07-28 17:11:27.240911884 +0000 UTC m=+69.841364231" Jul 28 17:11:27.330719 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:27.330585 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7b54dc8599-qn4bf_22ace845-d4cb-4788-b099-8b4990ca80e8/registry/0.log" Jul 28 17:11:27.667448 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:27.666859 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2e004692-883b-4311-acec-06713b142f67-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-t4kz8\" (UID: \"2e004692-883b-4311-acec-06713b142f67\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t4kz8" Jul 28 17:11:27.678757 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:27.673510 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2e004692-883b-4311-acec-06713b142f67-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-t4kz8\" (UID: \"2e004692-883b-4311-acec-06713b142f67\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t4kz8" Jul 28 17:11:27.860500 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:27.860254 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t4kz8" Jul 28 17:11:28.130523 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:28.130403 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w8nl6_ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d/node-ca/0.log" Jul 28 17:11:28.331054 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:28.331018 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d944fd6b9-xmtww_5d779727-dc91-4c3d-863a-622cbfdfbd47/router/0.log" Jul 28 17:11:28.640967 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:28.640919 2562 patch_prober.go:28] interesting pod/image-registry-7b54dc8599-qn4bf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 28 17:11:28.641269 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:28.640985 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" podUID="22ace845-d4cb-4788-b099-8b4990ca80e8" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:11:28.676776 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:28.676739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:28.677031 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:28.676790 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:28.677031 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:28.676870 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:11:28.677031 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:28.676918 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:11:28.677031 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:28.676963 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm podName:ec7b5a58-1942-455b-8195-c72e4c57e316 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:32.676941427 +0000 UTC m=+75.277393751 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s7zwq" (UID: "ec7b5a58-1942-455b-8195-c72e4c57e316") : configmap "kube-rbac-proxy" not found Jul 28 17:11:28.677031 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:28.676979 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls podName:ec7b5a58-1942-455b-8195-c72e4c57e316 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:32.676972683 +0000 UTC m=+75.277425007 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s7zwq" (UID: "ec7b5a58-1942-455b-8195-c72e4c57e316") : secret "insights-runtime-extractor-tls" not found Jul 28 17:11:28.930595 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:28.930510 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qqpxl_c0218446-ab4e-4002-9f70-1d5c5c222bed/serve-healthcheck-canary/0.log" Jul 28 17:11:30.152005 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:30.151974 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:11:30.459390 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:30.459228 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t4kz8"] Jul 28 17:11:30.462288 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:30.462256 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e004692_883b_4311_acec_06713b142f67.slice/crio-bfcbe0e069f68d09512c4fbcad86a0c28ebb480110522d727de8b990dbf29ed5 WatchSource:0}: Error finding container bfcbe0e069f68d09512c4fbcad86a0c28ebb480110522d727de8b990dbf29ed5: Status 404 returned error can't find the container with id bfcbe0e069f68d09512c4fbcad86a0c28ebb480110522d727de8b990dbf29ed5 Jul 28 17:11:31.231787 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:31.231656 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xn76m" event={"ID":"4f49cd3a-ee50-43ee-a297-ca95c775c799","Type":"ContainerStarted","Data":"cde8d315b024730bb66c5921c576afacf1953d7cbfe00a9ccb542f503a02e4e5"} Jul 28 17:11:31.232932 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:31.232903 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t4kz8" event={"ID":"2e004692-883b-4311-acec-06713b142f67","Type":"ContainerStarted","Data":"bfcbe0e069f68d09512c4fbcad86a0c28ebb480110522d727de8b990dbf29ed5"} Jul 28 17:11:31.234561 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:31.234533 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mpsgl" event={"ID":"1ca5eb66-7139-4339-82cb-e0c5cabb18f4","Type":"ContainerStarted","Data":"cc4c2f787cf85099e426073376f8f474f9da27df4902f3433306b816200c7a60"} Jul 28 17:11:31.234683 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:31.234672 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:11:31.249678 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:31.249632 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xn76m" podStartSLOduration=67.706411831 podStartE2EDuration="1m13.249617056s" totalImagesPullingTime="5.543205225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:31.248547742 +0000 UTC m=+73.849000100" watchObservedRunningTime="2026-07-28 17:11:31.249617056 +0000 UTC m=+73.850069430" Jul 28 17:11:31.270617 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:31.270545 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mpsgl" podStartSLOduration=67.735355669 podStartE2EDuration="1m13.270520949s" totalImagesPullingTime="5.53516528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:31.268965904 +0000 UTC m=+73.869418250" watchObservedRunningTime="2026-07-28 17:11:31.270520949 +0000 UTC m=+73.870973295" Jul 28 17:11:32.240482 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:32.240434 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t4kz8" event={"ID":"2e004692-883b-4311-acec-06713b142f67","Type":"ContainerStarted","Data":"2b92ed63ebec58a8e08a4eb528ebb59a480277c421fdc7e71519f3de153a8759"} Jul 28 17:11:32.267501 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:32.267442 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t4kz8" podStartSLOduration=4.820566135 podStartE2EDuration="6.267420273s" totalImagesPullingTime="1.446854138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:32.265493821 +0000 UTC m=+74.865946182" watchObservedRunningTime="2026-07-28 17:11:32.267420273 +0000 UTC m=+74.867872611" Jul 28 17:11:32.711129 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:32.711043 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:32.711129 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:32.711112 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:32.711315 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:32.711178 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:11:32.711315 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:32.711210 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:11:32.711315 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:32.711242 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm podName:ec7b5a58-1942-455b-8195-c72e4c57e316 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:40.711227774 +0000 UTC m=+83.311680097 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s7zwq" (UID: "ec7b5a58-1942-455b-8195-c72e4c57e316") : configmap "kube-rbac-proxy" not found Jul 28 17:11:32.711315 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:32.711276 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls podName:ec7b5a58-1942-455b-8195-c72e4c57e316 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:40.711263189 +0000 UTC m=+83.311715512 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s7zwq" (UID: "ec7b5a58-1942-455b-8195-c72e4c57e316") : secret "insights-runtime-extractor-tls" not found Jul 28 17:11:33.245055 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:33.244987 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t4kz8" Jul 28 17:11:33.250553 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:33.250526 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t4kz8" Jul 28 17:11:33.969980 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:33.969943 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-65k7t"] Jul 28 17:11:33.972667 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:33.972646 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" Jul 28 17:11:33.975742 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:33.975717 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 17:11:33.975906 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:33.975867 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 28 17:11:33.976035 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:33.975987 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 28 17:11:33.977308 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:33.977288 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 17:11:33.977412 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:33.977374 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 17:11:33.977558 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:33.977537 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-8pqkd\"" Jul 28 17:11:33.983614 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:33.983590 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-65k7t"] Jul 28 17:11:34.121220 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:34.121178 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e160931f-1ab3-488c-8788-ad4b0ca2083e-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-65k7t\" (UID: \"e160931f-1ab3-488c-8788-ad4b0ca2083e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" Jul 28 17:11:34.121416 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:34.121287 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e160931f-1ab3-488c-8788-ad4b0ca2083e-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-65k7t\" (UID: \"e160931f-1ab3-488c-8788-ad4b0ca2083e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" Jul 28 17:11:34.121416 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:34.121352 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e160931f-1ab3-488c-8788-ad4b0ca2083e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-65k7t\" (UID: \"e160931f-1ab3-488c-8788-ad4b0ca2083e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" Jul 28 17:11:34.121416 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:34.121384 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr9jw\" (UniqueName: \"kubernetes.io/projected/e160931f-1ab3-488c-8788-ad4b0ca2083e-kube-api-access-xr9jw\") pod \"prometheus-operator-84d69f794d-65k7t\" (UID: \"e160931f-1ab3-488c-8788-ad4b0ca2083e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" Jul 28 17:11:34.222013 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:34.221928 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e160931f-1ab3-488c-8788-ad4b0ca2083e-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-65k7t\" (UID: \"e160931f-1ab3-488c-8788-ad4b0ca2083e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" Jul 28 17:11:34.222013 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:34.221989 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e160931f-1ab3-488c-8788-ad4b0ca2083e-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-65k7t\" (UID: \"e160931f-1ab3-488c-8788-ad4b0ca2083e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" Jul 28 17:11:34.222218 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:34.222037 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e160931f-1ab3-488c-8788-ad4b0ca2083e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-65k7t\" (UID: \"e160931f-1ab3-488c-8788-ad4b0ca2083e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" Jul 28 17:11:34.222218 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:34.222061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr9jw\" (UniqueName: \"kubernetes.io/projected/e160931f-1ab3-488c-8788-ad4b0ca2083e-kube-api-access-xr9jw\") pod \"prometheus-operator-84d69f794d-65k7t\" (UID: \"e160931f-1ab3-488c-8788-ad4b0ca2083e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" Jul 28 17:11:34.222218 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:34.222177 2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 28 17:11:34.222368 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:34.222284 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e160931f-1ab3-488c-8788-ad4b0ca2083e-prometheus-operator-tls podName:e160931f-1ab3-488c-8788-ad4b0ca2083e nodeName:}" failed. No retries permitted until 2026-07-28 17:11:34.722260459 +0000 UTC m=+77.322712797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/e160931f-1ab3-488c-8788-ad4b0ca2083e-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-65k7t" (UID: "e160931f-1ab3-488c-8788-ad4b0ca2083e") : secret "prometheus-operator-tls" not found Jul 28 17:11:34.222737 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:34.222712 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e160931f-1ab3-488c-8788-ad4b0ca2083e-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-65k7t\" (UID: \"e160931f-1ab3-488c-8788-ad4b0ca2083e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" Jul 28 17:11:34.224598 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:34.224576 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e160931f-1ab3-488c-8788-ad4b0ca2083e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-65k7t\" (UID: \"e160931f-1ab3-488c-8788-ad4b0ca2083e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" Jul 28 17:11:34.232686 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:34.232660 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr9jw\" (UniqueName: \"kubernetes.io/projected/e160931f-1ab3-488c-8788-ad4b0ca2083e-kube-api-access-xr9jw\") pod \"prometheus-operator-84d69f794d-65k7t\" (UID: \"e160931f-1ab3-488c-8788-ad4b0ca2083e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" Jul 28 17:11:34.727502 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:34.727462 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e160931f-1ab3-488c-8788-ad4b0ca2083e-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-65k7t\" (UID: \"e160931f-1ab3-488c-8788-ad4b0ca2083e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" Jul 28 17:11:34.730089 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:34.730064 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e160931f-1ab3-488c-8788-ad4b0ca2083e-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-65k7t\" (UID: \"e160931f-1ab3-488c-8788-ad4b0ca2083e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" Jul 28 17:11:34.884788 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:34.884747 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" Jul 28 17:11:35.031469 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:35.031378 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-65k7t"] Jul 28 17:11:35.036295 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:35.036264 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode160931f_1ab3_488c_8788_ad4b0ca2083e.slice/crio-3a2273d20b1cc9001ee970828582dc39b1419a39c54d217fa55c5ec7180e863a WatchSource:0}: Error finding container 3a2273d20b1cc9001ee970828582dc39b1419a39c54d217fa55c5ec7180e863a: Status 404 returned error can't find the container with id 3a2273d20b1cc9001ee970828582dc39b1419a39c54d217fa55c5ec7180e863a Jul 28 17:11:35.253466 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:35.253422 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" event={"ID":"e160931f-1ab3-488c-8788-ad4b0ca2083e","Type":"ContainerStarted","Data":"3a2273d20b1cc9001ee970828582dc39b1419a39c54d217fa55c5ec7180e863a"} Jul 28 17:11:37.263386 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:37.263338 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" event={"ID":"e160931f-1ab3-488c-8788-ad4b0ca2083e","Type":"ContainerStarted","Data":"6c4bc227950c947fa49ca0a87552cf782cad4a6161acc2b4b4c7391ff489b489"} Jul 28 17:11:37.263780 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:37.263461 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" event={"ID":"e160931f-1ab3-488c-8788-ad4b0ca2083e","Type":"ContainerStarted","Data":"b893e919bba4cc1e96c02f4642759209fb6a3beabc6a1a9467ce26aedc224b20"} Jul 28 17:11:37.298260 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:37.298201 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-65k7t" podStartSLOduration=3.07707384 podStartE2EDuration="4.298184234s" totalImagesPullingTime="1.221110394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:37.297317761 +0000 UTC m=+79.897770107" watchObservedRunningTime="2026-07-28 17:11:37.298184234 +0000 UTC m=+79.898636581" Jul 28 17:11:39.331012 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.330977 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h"] Jul 28 17:11:39.336292 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.336266 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" Jul 28 17:11:39.339208 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.339181 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 28 17:11:39.339381 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.339251 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 28 17:11:39.339481 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.339464 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bm9vs\"" Jul 28 17:11:39.351604 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.350982 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h"] Jul 28 17:11:39.351761 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.351605 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5"] Jul 28 17:11:39.359564 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.359542 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wspw8"] Jul 28 17:11:39.359725 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.359590 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.363678 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.363658 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.366676 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.363969 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 28 17:11:39.366676 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.366377 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5"] Jul 28 17:11:39.366676 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.366538 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 28 17:11:39.368856 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.367073 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-tfm6z\"" Jul 28 17:11:39.368856 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.367210 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 28 17:11:39.368856 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.367371 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 28 17:11:39.368856 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.367435 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-c44q6\"" Jul 28 17:11:39.368856 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.367554 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 28 17:11:39.368856 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.367653 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 28 17:11:39.474079 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474038 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7d976b-5549-4246-be0b-64772c7ca3a8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-2wj2h\" (UID: \"4a7d976b-5549-4246-be0b-64772c7ca3a8\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" Jul 28 17:11:39.474277 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474097 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-textfile\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.474277 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474127 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-wtmp\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.474277 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474173 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-sys\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.474277 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474203 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/67735dc9-f5eb-4144-9814-9e899af24e05-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.474521 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474288 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/67735dc9-f5eb-4144-9814-9e899af24e05-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.474521 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474339 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxb4j\" (UniqueName: \"kubernetes.io/projected/4a7d976b-5549-4246-be0b-64772c7ca3a8-kube-api-access-rxb4j\") pod \"openshift-state-metrics-b8f49f5c6-2wj2h\" (UID: \"4a7d976b-5549-4246-be0b-64772c7ca3a8\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" Jul 28 17:11:39.474521 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474372 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwx4t\" (UniqueName: \"kubernetes.io/projected/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-kube-api-access-gwx4t\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.474521 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474426 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdgf2\" (UniqueName: \"kubernetes.io/projected/67735dc9-f5eb-4144-9814-9e899af24e05-kube-api-access-cdgf2\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.474521 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474471 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67735dc9-f5eb-4144-9814-9e899af24e05-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.474764 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474522 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-metrics-client-ca\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.474764 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474548 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-root\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.474764 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474576 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-accelerators-collector-config\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.474764 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474606 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4a7d976b-5549-4246-be0b-64772c7ca3a8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-2wj2h\" (UID: \"4a7d976b-5549-4246-be0b-64772c7ca3a8\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" Jul 28 17:11:39.474764 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474636 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.474764 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474662 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a7d976b-5549-4246-be0b-64772c7ca3a8-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-2wj2h\" (UID: \"4a7d976b-5549-4246-be0b-64772c7ca3a8\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" Jul 28 17:11:39.474764 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474707 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67735dc9-f5eb-4144-9814-9e899af24e05-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.474764 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474737 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/67735dc9-f5eb-4144-9814-9e899af24e05-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.475138 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.474768 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-tls\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.578333 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578294 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67735dc9-f5eb-4144-9814-9e899af24e05-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.578602 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578358 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-metrics-client-ca\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.578602 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578384 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-root\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.578602 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578408 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-accelerators-collector-config\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.578602 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578441 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4a7d976b-5549-4246-be0b-64772c7ca3a8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-2wj2h\" (UID: \"4a7d976b-5549-4246-be0b-64772c7ca3a8\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" Jul 28 17:11:39.578602 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578473 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.578602 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578497 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a7d976b-5549-4246-be0b-64772c7ca3a8-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-2wj2h\" (UID: \"4a7d976b-5549-4246-be0b-64772c7ca3a8\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" Jul 28 17:11:39.578602 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67735dc9-f5eb-4144-9814-9e899af24e05-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.578602 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578561 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/67735dc9-f5eb-4144-9814-9e899af24e05-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.578602 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578588 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-tls\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.578889 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578616 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7d976b-5549-4246-be0b-64772c7ca3a8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-2wj2h\" (UID: \"4a7d976b-5549-4246-be0b-64772c7ca3a8\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" Jul 28 17:11:39.578889 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578647 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-textfile\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.578889 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578675 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-wtmp\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.578889 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578726 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-sys\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.578889 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578762 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/67735dc9-f5eb-4144-9814-9e899af24e05-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.578889 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578795 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/67735dc9-f5eb-4144-9814-9e899af24e05-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.578889 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578819 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxb4j\" (UniqueName: \"kubernetes.io/projected/4a7d976b-5549-4246-be0b-64772c7ca3a8-kube-api-access-rxb4j\") pod \"openshift-state-metrics-b8f49f5c6-2wj2h\" (UID: \"4a7d976b-5549-4246-be0b-64772c7ca3a8\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" Jul 28 17:11:39.578889 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578853 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwx4t\" (UniqueName: \"kubernetes.io/projected/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-kube-api-access-gwx4t\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.579120 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.578906 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdgf2\" (UniqueName: \"kubernetes.io/projected/67735dc9-f5eb-4144-9814-9e899af24e05-kube-api-access-cdgf2\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.580758 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:39.579397 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 28 17:11:39.580758 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:11:39.579466 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-tls podName:d3159c85-7f2a-4e8a-b074-e177e9dbfd12 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:40.079445821 +0000 UTC m=+82.679898146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-tls") pod "node-exporter-wspw8" (UID: "d3159c85-7f2a-4e8a-b074-e177e9dbfd12") : secret "node-exporter-tls" not found Jul 28 17:11:39.580758 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.580041 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/67735dc9-f5eb-4144-9814-9e899af24e05-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.580758 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.580248 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-wtmp\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.580758 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.580269 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-textfile\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.580758 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.580319 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-sys\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.596438 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.586438 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4a7d976b-5549-4246-be0b-64772c7ca3a8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-2wj2h\" (UID: \"4a7d976b-5549-4246-be0b-64772c7ca3a8\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" Jul 28 17:11:39.596438 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.586504 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-accelerators-collector-config\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.596438 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.586800 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-metrics-client-ca\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.596438 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.587060 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-root\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.596438 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.588581 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67735dc9-f5eb-4144-9814-9e899af24e05-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.596438 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.592272 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/67735dc9-f5eb-4144-9814-9e899af24e05-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.596438 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.593893 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a7d976b-5549-4246-be0b-64772c7ca3a8-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-2wj2h\" (UID: \"4a7d976b-5549-4246-be0b-64772c7ca3a8\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" Jul 28 17:11:39.596438 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.594598 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a7d976b-5549-4246-be0b-64772c7ca3a8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-2wj2h\" (UID: \"4a7d976b-5549-4246-be0b-64772c7ca3a8\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" Jul 28 17:11:39.596438 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.595345 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.596438 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.595719 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/67735dc9-f5eb-4144-9814-9e899af24e05-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.598744 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.598590 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwx4t\" (UniqueName: \"kubernetes.io/projected/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-kube-api-access-gwx4t\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:39.598744 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.598700 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxb4j\" (UniqueName: \"kubernetes.io/projected/4a7d976b-5549-4246-be0b-64772c7ca3a8-kube-api-access-rxb4j\") pod \"openshift-state-metrics-b8f49f5c6-2wj2h\" (UID: \"4a7d976b-5549-4246-be0b-64772c7ca3a8\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" Jul 28 17:11:39.598954 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.598912 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67735dc9-f5eb-4144-9814-9e899af24e05-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.600595 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.600561 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdgf2\" (UniqueName: \"kubernetes.io/projected/67735dc9-f5eb-4144-9814-9e899af24e05-kube-api-access-cdgf2\") pod \"kube-state-metrics-56d9f944b4-sxlc5\" (UID: \"67735dc9-f5eb-4144-9814-9e899af24e05\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:39.652357 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.652319 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" Jul 28 17:11:39.679762 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:39.679585 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" Jul 28 17:11:40.083735 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:40.083698 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-tls\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:40.086239 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:40.086208 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d3159c85-7f2a-4e8a-b074-e177e9dbfd12-node-exporter-tls\") pod \"node-exporter-wspw8\" (UID: \"d3159c85-7f2a-4e8a-b074-e177e9dbfd12\") " pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:40.286214 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:40.286185 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wspw8" Jul 28 17:11:40.790434 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:40.790392 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:40.790859 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:40.790475 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:40.791044 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:40.791003 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec7b5a58-1942-455b-8195-c72e4c57e316-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:40.793039 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:40.793013 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec7b5a58-1942-455b-8195-c72e4c57e316-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s7zwq\" (UID: \"ec7b5a58-1942-455b-8195-c72e4c57e316\") " pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:40.872131 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:40.872091 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s7zwq" Jul 28 17:11:43.676553 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.676513 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5c779bb47b-8tx5c"] Jul 28 17:11:43.679775 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.679748 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.683405 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.683381 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 28 17:11:43.684985 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.684830 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 28 17:11:43.684985 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.684836 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2twmw\"" Jul 28 17:11:43.684985 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.684830 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 28 17:11:43.684985 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.684836 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1lhck47853esg\"" Jul 28 17:11:43.684985 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.684928 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 28 17:11:43.692374 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.692350 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c779bb47b-8tx5c"] Jul 28 17:11:43.820824 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.820776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7765c642-f896-4490-bddc-5c10f97fc55a-secret-metrics-server-tls\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.821007 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.820845 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7765c642-f896-4490-bddc-5c10f97fc55a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.821007 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.820896 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xldm4\" (UniqueName: \"kubernetes.io/projected/7765c642-f896-4490-bddc-5c10f97fc55a-kube-api-access-xldm4\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.821007 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.820988 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7765c642-f896-4490-bddc-5c10f97fc55a-metrics-server-audit-profiles\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.821137 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.821049 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7765c642-f896-4490-bddc-5c10f97fc55a-secret-metrics-server-client-certs\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.821137 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.821078 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7765c642-f896-4490-bddc-5c10f97fc55a-audit-log\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.821210 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.821155 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7765c642-f896-4490-bddc-5c10f97fc55a-client-ca-bundle\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.922311 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.922269 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7765c642-f896-4490-bddc-5c10f97fc55a-client-ca-bundle\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.922503 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.922322 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7765c642-f896-4490-bddc-5c10f97fc55a-secret-metrics-server-tls\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.922503 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.922361 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7765c642-f896-4490-bddc-5c10f97fc55a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.922503 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.922385 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xldm4\" (UniqueName: \"kubernetes.io/projected/7765c642-f896-4490-bddc-5c10f97fc55a-kube-api-access-xldm4\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.922503 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.922439 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7765c642-f896-4490-bddc-5c10f97fc55a-metrics-server-audit-profiles\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.922503 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.922477 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7765c642-f896-4490-bddc-5c10f97fc55a-secret-metrics-server-client-certs\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.922503 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.922501 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7765c642-f896-4490-bddc-5c10f97fc55a-audit-log\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.922906 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.922858 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7765c642-f896-4490-bddc-5c10f97fc55a-audit-log\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.923573 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.923543 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7765c642-f896-4490-bddc-5c10f97fc55a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.923809 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.923785 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7765c642-f896-4490-bddc-5c10f97fc55a-metrics-server-audit-profiles\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.925631 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.925607 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7765c642-f896-4490-bddc-5c10f97fc55a-client-ca-bundle\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.925730 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.925680 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7765c642-f896-4490-bddc-5c10f97fc55a-secret-metrics-server-client-certs\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.925730 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.925702 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7765c642-f896-4490-bddc-5c10f97fc55a-secret-metrics-server-tls\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.932160 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.932096 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xldm4\" (UniqueName: \"kubernetes.io/projected/7765c642-f896-4490-bddc-5c10f97fc55a-kube-api-access-xldm4\") pod \"metrics-server-5c779bb47b-8tx5c\" (UID: \"7765c642-f896-4490-bddc-5c10f97fc55a\") " pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:43.990486 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:43.990440 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:11:44.295235 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:44.295198 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b54dc8599-qn4bf"] Jul 28 17:11:44.956184 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:44.956138 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3159c85_7f2a_4e8a_b074_e177e9dbfd12.slice/crio-39a0ba7c24184577cf629f808d78d68bd44c67d1f4c2c2f3981419b44e092856 WatchSource:0}: Error finding container 39a0ba7c24184577cf629f808d78d68bd44c67d1f4c2c2f3981419b44e092856: Status 404 returned error can't find the container with id 39a0ba7c24184577cf629f808d78d68bd44c67d1f4c2c2f3981419b44e092856 Jul 28 17:11:45.108979 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.108953 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5"] Jul 28 17:11:45.156315 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.156283 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c779bb47b-8tx5c"] Jul 28 17:11:45.191217 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:45.191172 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67735dc9_f5eb_4144_9814_9e899af24e05.slice/crio-e7b067231014370fe3a9d8e4b3a74cba0d9e10f549a1d95414b101a482255463 WatchSource:0}: Error finding container e7b067231014370fe3a9d8e4b3a74cba0d9e10f549a1d95414b101a482255463: Status 404 returned error can't find the container with id e7b067231014370fe3a9d8e4b3a74cba0d9e10f549a1d95414b101a482255463 Jul 28 17:11:45.191871 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:45.191839 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7765c642_f896_4490_bddc_5c10f97fc55a.slice/crio-8e0f1802c4e4a295fbc24f4ec2d4c76caa41f851995360a98ecae613300c413e WatchSource:0}: Error finding container 8e0f1802c4e4a295fbc24f4ec2d4c76caa41f851995360a98ecae613300c413e: Status 404 returned error can't find the container with id 8e0f1802c4e4a295fbc24f4ec2d4c76caa41f851995360a98ecae613300c413e Jul 28 17:11:45.295938 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.295905 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wspw8" event={"ID":"d3159c85-7f2a-4e8a-b074-e177e9dbfd12","Type":"ContainerStarted","Data":"39a0ba7c24184577cf629f808d78d68bd44c67d1f4c2c2f3981419b44e092856"} Jul 28 17:11:45.297214 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.297181 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" event={"ID":"67735dc9-f5eb-4144-9814-9e899af24e05","Type":"ContainerStarted","Data":"e7b067231014370fe3a9d8e4b3a74cba0d9e10f549a1d95414b101a482255463"} Jul 28 17:11:45.298426 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.298404 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" event={"ID":"7765c642-f896-4490-bddc-5c10f97fc55a","Type":"ContainerStarted","Data":"8e0f1802c4e4a295fbc24f4ec2d4c76caa41f851995360a98ecae613300c413e"} Jul 28 17:11:45.331219 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.330926 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s7zwq"] Jul 28 17:11:45.332929 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.332902 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h"] Jul 28 17:11:45.334206 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:45.334130 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec7b5a58_1942_455b_8195_c72e4c57e316.slice/crio-43f7d42727ce36382bee01d77c00d1116144847a9c027c476b93531ed1e05779 WatchSource:0}: Error finding container 43f7d42727ce36382bee01d77c00d1116144847a9c027c476b93531ed1e05779: Status 404 returned error can't find the container with id 43f7d42727ce36382bee01d77c00d1116144847a9c027c476b93531ed1e05779 Jul 28 17:11:45.343067 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:45.343041 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a7d976b_5549_4246_be0b_64772c7ca3a8.slice/crio-e2c0c632537f7b40db4e4675004c343fdbeb2afba6b33f5bd5e105992775f61e WatchSource:0}: Error finding container e2c0c632537f7b40db4e4675004c343fdbeb2afba6b33f5bd5e105992775f61e: Status 404 returned error can't find the container with id e2c0c632537f7b40db4e4675004c343fdbeb2afba6b33f5bd5e105992775f61e Jul 28 17:11:45.587123 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.587092 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:11:45.609446 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.609410 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.614729 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.614703 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 28 17:11:45.615012 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.614978 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 28 17:11:45.615665 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.615145 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 28 17:11:45.615665 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.615195 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 28 17:11:45.615665 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.615217 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 28 17:11:45.615665 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.615287 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 28 17:11:45.615665 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.615321 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 28 17:11:45.615665 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.615397 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 28 17:11:45.615665 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.615418 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 28 17:11:45.615665 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.615466 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 28 17:11:45.616684 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.616354 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7lvu1lj77bi9d\"" Jul 28 17:11:45.616684 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.616546 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 28 17:11:45.617049 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.616855 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 28 17:11:45.617049 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.616949 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-cnkp5\"" Jul 28 17:11:45.617049 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.617013 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 28 17:11:45.617340 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.617321 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 28 17:11:45.627105 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.625341 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:11:45.738581 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.738548 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cdd06f73-9108-4ae6-88db-6d661dc3be64-config-out\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.738764 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.738591 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.738764 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.738628 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.738764 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.738666 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.738764 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.738690 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-web-config\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.738764 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.738709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.738764 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.738734 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwxlb\" (UniqueName: \"kubernetes.io/projected/cdd06f73-9108-4ae6-88db-6d661dc3be64-kube-api-access-vwxlb\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.739103 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.738796 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.739103 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.738930 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.739103 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.739075 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.739252 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.739117 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.739252 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.739153 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.739252 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.739186 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cdd06f73-9108-4ae6-88db-6d661dc3be64-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.739252 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.739215 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-config\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.739252 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.739241 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.739492 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.739282 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.739492 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.739313 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.739492 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.739341 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.739492 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.739370 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.739492 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.739399 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.840916 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.840536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.840916 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.840599 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.840916 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.840720 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.840916 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.840753 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.840916 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.840793 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cdd06f73-9108-4ae6-88db-6d661dc3be64-config-out\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.840916 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.840827 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.840916 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.840859 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.840916 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.840921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.841452 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.840946 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-web-config\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.841452 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.840971 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.841452 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.840999 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwxlb\" (UniqueName: \"kubernetes.io/projected/cdd06f73-9108-4ae6-88db-6d661dc3be64-kube-api-access-vwxlb\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.841452 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.841039 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.841452 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.841079 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.841452 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.841116 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.841452 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.841158 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.841452 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.841196 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.841452 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.841272 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cdd06f73-9108-4ae6-88db-6d661dc3be64-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.841452 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.841303 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-config\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.841452 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.841332 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.841452 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.841371 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.843651 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.842952 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.843651 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.843458 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.850452 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.850161 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.855804 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.852630 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.855804 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.854568 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.855804 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.855739 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cdd06f73-9108-4ae6-88db-6d661dc3be64-config-out\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.858718 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.857353 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.858718 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.858087 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.867433 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.866057 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.867433 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.866754 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.871373 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.870687 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.871373 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.871307 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.872447 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.872357 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-config\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.891271 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.891227 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.891453 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.891233 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.891453 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.891413 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwxlb\" (UniqueName: \"kubernetes.io/projected/cdd06f73-9108-4ae6-88db-6d661dc3be64-kube-api-access-vwxlb\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.893734 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.893706 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cdd06f73-9108-4ae6-88db-6d661dc3be64-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.895905 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.895857 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.896597 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.896569 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.897796 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.897770 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-web-config\") pod \"prometheus-k8s-0\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:45.928662 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:45.928328 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:11:46.308098 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:46.308053 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" event={"ID":"4a7d976b-5549-4246-be0b-64772c7ca3a8","Type":"ContainerStarted","Data":"ad5e9974796f83d1a307b6acb497c84b1adf0df6e06f87178e68b3db5d83b147"} Jul 28 17:11:46.308640 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:46.308135 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" event={"ID":"4a7d976b-5549-4246-be0b-64772c7ca3a8","Type":"ContainerStarted","Data":"8d9b0dfd1c304e01dd6e15572b8bdce0b38ab9c561a1a65a97ac4427d693bddd"} Jul 28 17:11:46.308640 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:46.308165 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" event={"ID":"4a7d976b-5549-4246-be0b-64772c7ca3a8","Type":"ContainerStarted","Data":"e2c0c632537f7b40db4e4675004c343fdbeb2afba6b33f5bd5e105992775f61e"} Jul 28 17:11:46.311005 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:46.310229 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-hblt4" event={"ID":"86e8f2ce-1d58-4acc-8d6e-fbb4bc8d208c","Type":"ContainerStarted","Data":"99dfc4546a62b3f9755035eb95681e74983eb55e7390cf47db1b0c1a35f25b8a"} Jul 28 17:11:46.311005 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:46.310675 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-hblt4" Jul 28 17:11:46.312888 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:46.312809 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s7zwq" event={"ID":"ec7b5a58-1942-455b-8195-c72e4c57e316","Type":"ContainerStarted","Data":"43f7d42727ce36382bee01d77c00d1116144847a9c027c476b93531ed1e05779"} Jul 28 17:11:46.320165 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:46.320140 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-hblt4" Jul 28 17:11:46.352909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:46.351655 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-hblt4" podStartSLOduration=1.829279506 podStartE2EDuration="20.351632625s" totalImagesPullingTime="18.522353119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:46.330028664 +0000 UTC m=+88.930481009" watchObservedRunningTime="2026-07-28 17:11:46.351632625 +0000 UTC m=+88.952084972" Jul 28 17:11:46.625947 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:46.625596 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:11:46.819613 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:11:46.819568 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdd06f73_9108_4ae6_88db_6d661dc3be64.slice/crio-427a0a2e706d69ec45aeec19c550694f0c81a8d07b395848e8ef3c0b90eef575 WatchSource:0}: Error finding container 427a0a2e706d69ec45aeec19c550694f0c81a8d07b395848e8ef3c0b90eef575: Status 404 returned error can't find the container with id 427a0a2e706d69ec45aeec19c550694f0c81a8d07b395848e8ef3c0b90eef575 Jul 28 17:11:47.322822 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:47.322785 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdd06f73-9108-4ae6-88db-6d661dc3be64","Type":"ContainerStarted","Data":"427a0a2e706d69ec45aeec19c550694f0c81a8d07b395848e8ef3c0b90eef575"} Jul 28 17:11:49.356737 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:49.356643 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wspw8" event={"ID":"d3159c85-7f2a-4e8a-b074-e177e9dbfd12","Type":"ContainerStarted","Data":"7c0682deb5931fd8283dfdefa9d0714130db06da8379875ba7a59639410cb3e3"} Jul 28 17:11:49.364533 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:49.364458 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" event={"ID":"7765c642-f896-4490-bddc-5c10f97fc55a","Type":"ContainerStarted","Data":"8d25ea2e664065bbb239b6584872f2ee487dbcffedbfb276956b74e61c72b17e"} Jul 28 17:11:49.375692 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:49.373345 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" event={"ID":"67735dc9-f5eb-4144-9814-9e899af24e05","Type":"ContainerStarted","Data":"8d14f8a36a17c670cb5f6c1a8e4c68aab598d484206148b7fb6c702065e3a116"} Jul 28 17:11:49.408421 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:49.407902 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" podStartSLOduration=2.530788275 podStartE2EDuration="6.407854375s" totalImagesPullingTime="3.8770661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:49.397373409 +0000 UTC m=+91.997825756" watchObservedRunningTime="2026-07-28 17:11:49.407854375 +0000 UTC m=+92.008306722" Jul 28 17:11:50.380349 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:50.380306 2562 generic.go:356] "Generic (PLEG): container finished" podID="d3159c85-7f2a-4e8a-b074-e177e9dbfd12" containerID="7c0682deb5931fd8283dfdefa9d0714130db06da8379875ba7a59639410cb3e3" exitCode=0 Jul 28 17:11:50.380815 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:50.380471 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wspw8" event={"ID":"d3159c85-7f2a-4e8a-b074-e177e9dbfd12","Type":"ContainerDied","Data":"7c0682deb5931fd8283dfdefa9d0714130db06da8379875ba7a59639410cb3e3"} Jul 28 17:11:50.383506 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:50.383475 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" event={"ID":"67735dc9-f5eb-4144-9814-9e899af24e05","Type":"ContainerStarted","Data":"7bcbb745919e2099f56e7a14f6ec85dce3b88e9de713b4864e78cfd3d3ee0f3a"} Jul 28 17:11:50.383653 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:50.383539 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" event={"ID":"67735dc9-f5eb-4144-9814-9e899af24e05","Type":"ContainerStarted","Data":"77a23594cb2203a1c45296cfc9f107c4ce0fd400d68d4fd45de4428a963dad25"} Jul 28 17:11:50.425273 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:50.425213 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-sxlc5" podStartSLOduration=7.557099416 podStartE2EDuration="11.425193497s" totalImagesPullingTime="3.868094081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:50.423894013 +0000 UTC m=+93.024346351" watchObservedRunningTime="2026-07-28 17:11:50.425193497 +0000 UTC m=+93.025645842" Jul 28 17:11:55.414266 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:55.414224 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" event={"ID":"4a7d976b-5549-4246-be0b-64772c7ca3a8","Type":"ContainerStarted","Data":"dbb514a8c0d209e2e4b3ad27c704ba6566d7af5e1e546a0d429d16f5da1468a4"} Jul 28 17:11:55.416564 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:55.416533 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wspw8" event={"ID":"d3159c85-7f2a-4e8a-b074-e177e9dbfd12","Type":"ContainerStarted","Data":"960f8d07568dcba4fb0e571df698cf3e2b8fa72c56810e1205e26f08fced5f61"} Jul 28 17:11:55.416717 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:55.416594 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wspw8" event={"ID":"d3159c85-7f2a-4e8a-b074-e177e9dbfd12","Type":"ContainerStarted","Data":"6450cae5302eae3d4f0a6370ac80390bf38e4dc4eae298cb55c5310770767681"} Jul 28 17:11:55.418345 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:55.418313 2562 generic.go:356] "Generic (PLEG): container finished" podID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerID="6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a" exitCode=0 Jul 28 17:11:55.418462 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:55.418373 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdd06f73-9108-4ae6-88db-6d661dc3be64","Type":"ContainerDied","Data":"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a"} Jul 28 17:11:55.436537 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:55.436493 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2wj2h" podStartSLOduration=7.143600212 podStartE2EDuration="16.436478836s" totalImagesPullingTime="9.292878624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:55.435514353 +0000 UTC m=+98.035966699" watchObservedRunningTime="2026-07-28 17:11:55.436478836 +0000 UTC m=+98.036931185" Jul 28 17:11:55.493386 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:55.493335 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wspw8" podStartSLOduration=12.163031989 podStartE2EDuration="16.493317853s" totalImagesPullingTime="4.116420053s" totalInitContainerRuntime="213.865811ms" isStatefulPod=true podCreationTimestamp="2026-07-28 17:11:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:55.491189558 +0000 UTC m=+98.091641904" watchObservedRunningTime="2026-07-28 17:11:55.493317853 +0000 UTC m=+98.093770199" Jul 28 17:11:56.435313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:11:56.434419 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s7zwq" event={"ID":"ec7b5a58-1942-455b-8195-c72e4c57e316","Type":"ContainerStarted","Data":"dffc30dc5ffc190ea3b003d10fca1c04a931cdd1918623a3dc36d658828a8a20"} Jul 28 17:12:01.459355 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:01.459320 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdd06f73-9108-4ae6-88db-6d661dc3be64","Type":"ContainerStarted","Data":"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492"} Jul 28 17:12:02.243189 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:02.243156 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mpsgl" Jul 28 17:12:02.467700 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:02.467654 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdd06f73-9108-4ae6-88db-6d661dc3be64","Type":"ContainerStarted","Data":"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad"} Jul 28 17:12:02.469823 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:02.469781 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s7zwq" event={"ID":"ec7b5a58-1942-455b-8195-c72e4c57e316","Type":"ContainerStarted","Data":"aa99d9dd283206448dcd81c50809efa3ecabe740a7f945c3ee52590c550bb1d8"} Jul 28 17:12:03.990711 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:03.990680 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:12:03.991181 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:03.990747 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:12:06.493032 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:06.492998 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s7zwq" event={"ID":"ec7b5a58-1942-455b-8195-c72e4c57e316","Type":"ContainerStarted","Data":"166feb91d4fe6fb28578c7497397236a0bf6738fd851329591d005fe2d61cb15"} Jul 28 17:12:06.496515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:06.496488 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdd06f73-9108-4ae6-88db-6d661dc3be64","Type":"ContainerStarted","Data":"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0"} Jul 28 17:12:06.496617 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:06.496562 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdd06f73-9108-4ae6-88db-6d661dc3be64","Type":"ContainerStarted","Data":"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96"} Jul 28 17:12:06.496617 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:06.496602 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdd06f73-9108-4ae6-88db-6d661dc3be64","Type":"ContainerStarted","Data":"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450"} Jul 28 17:12:06.514730 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:06.514689 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-s7zwq" podStartSLOduration=22.001852954 podStartE2EDuration="42.514675877s" totalImagesPullingTime="20.512822923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:11:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:12:06.513415509 +0000 UTC m=+109.113867856" watchObservedRunningTime="2026-07-28 17:12:06.514675877 +0000 UTC m=+109.115128224" Jul 28 17:12:07.503969 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:07.503930 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdd06f73-9108-4ae6-88db-6d661dc3be64","Type":"ContainerStarted","Data":"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0"} Jul 28 17:12:07.537266 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:07.537210 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.185361741 podStartE2EDuration="22.537188882s" totalImagesPullingTime="18.338534927s" totalInitContainerRuntime="13.292214ms" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:12:07.534710104 +0000 UTC m=+110.135162451" watchObservedRunningTime="2026-07-28 17:12:07.537188882 +0000 UTC m=+110.137641227" Jul 28 17:12:09.315682 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.315608 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" podUID="22ace845-d4cb-4788-b099-8b4990ca80e8" containerName="registry" containerID="cri-o://114f1c1e2a0ecccdf7430a4decf67d741c6092f4e8aefe4c0a598af40aa035ef" gracePeriod=30 Jul 28 17:12:09.514829 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.514786 2562 generic.go:356] "Generic (PLEG): container finished" podID="22ace845-d4cb-4788-b099-8b4990ca80e8" containerID="114f1c1e2a0ecccdf7430a4decf67d741c6092f4e8aefe4c0a598af40aa035ef" exitCode=0 Jul 28 17:12:09.515022 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.514936 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" event={"ID":"22ace845-d4cb-4788-b099-8b4990ca80e8","Type":"ContainerDied","Data":"114f1c1e2a0ecccdf7430a4decf67d741c6092f4e8aefe4c0a598af40aa035ef"} Jul 28 17:12:09.619303 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.619259 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:12:09.687651 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.687616 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/22ace845-d4cb-4788-b099-8b4990ca80e8-installation-pull-secrets\") pod \"22ace845-d4cb-4788-b099-8b4990ca80e8\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " Jul 28 17:12:09.687851 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.687666 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22ace845-d4cb-4788-b099-8b4990ca80e8-trusted-ca\") pod \"22ace845-d4cb-4788-b099-8b4990ca80e8\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " Jul 28 17:12:09.687851 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.687701 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-bound-sa-token\") pod \"22ace845-d4cb-4788-b099-8b4990ca80e8\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " Jul 28 17:12:09.687851 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.687729 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls\") pod \"22ace845-d4cb-4788-b099-8b4990ca80e8\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " Jul 28 17:12:09.687851 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.687764 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/22ace845-d4cb-4788-b099-8b4990ca80e8-image-registry-private-configuration\") pod \"22ace845-d4cb-4788-b099-8b4990ca80e8\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " Jul 28 17:12:09.687851 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.687793 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/22ace845-d4cb-4788-b099-8b4990ca80e8-ca-trust-extracted\") pod \"22ace845-d4cb-4788-b099-8b4990ca80e8\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " Jul 28 17:12:09.687851 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.687833 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxlrq\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-kube-api-access-cxlrq\") pod \"22ace845-d4cb-4788-b099-8b4990ca80e8\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " Jul 28 17:12:09.688200 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.687895 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-certificates\") pod \"22ace845-d4cb-4788-b099-8b4990ca80e8\" (UID: \"22ace845-d4cb-4788-b099-8b4990ca80e8\") " Jul 28 17:12:09.688254 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.688201 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22ace845-d4cb-4788-b099-8b4990ca80e8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "22ace845-d4cb-4788-b099-8b4990ca80e8" (UID: "22ace845-d4cb-4788-b099-8b4990ca80e8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:12:09.688487 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.688458 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "22ace845-d4cb-4788-b099-8b4990ca80e8" (UID: "22ace845-d4cb-4788-b099-8b4990ca80e8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:12:09.690940 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.690908 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "22ace845-d4cb-4788-b099-8b4990ca80e8" (UID: "22ace845-d4cb-4788-b099-8b4990ca80e8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:12:09.691106 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.690951 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22ace845-d4cb-4788-b099-8b4990ca80e8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "22ace845-d4cb-4788-b099-8b4990ca80e8" (UID: "22ace845-d4cb-4788-b099-8b4990ca80e8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:12:09.691173 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.691103 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "22ace845-d4cb-4788-b099-8b4990ca80e8" (UID: "22ace845-d4cb-4788-b099-8b4990ca80e8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:12:09.691173 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.691124 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22ace845-d4cb-4788-b099-8b4990ca80e8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "22ace845-d4cb-4788-b099-8b4990ca80e8" (UID: "22ace845-d4cb-4788-b099-8b4990ca80e8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:12:09.691278 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.691213 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-kube-api-access-cxlrq" (OuterVolumeSpecName: "kube-api-access-cxlrq") pod "22ace845-d4cb-4788-b099-8b4990ca80e8" (UID: "22ace845-d4cb-4788-b099-8b4990ca80e8"). InnerVolumeSpecName "kube-api-access-cxlrq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:12:09.700250 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.700218 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22ace845-d4cb-4788-b099-8b4990ca80e8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "22ace845-d4cb-4788-b099-8b4990ca80e8" (UID: "22ace845-d4cb-4788-b099-8b4990ca80e8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:12:09.788743 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.788700 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/22ace845-d4cb-4788-b099-8b4990ca80e8-installation-pull-secrets\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:12:09.788743 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.788744 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22ace845-d4cb-4788-b099-8b4990ca80e8-trusted-ca\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:12:09.788975 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.788758 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-bound-sa-token\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:12:09.788975 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.788771 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-tls\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:12:09.788975 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.788784 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/22ace845-d4cb-4788-b099-8b4990ca80e8-image-registry-private-configuration\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:12:09.788975 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.788799 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/22ace845-d4cb-4788-b099-8b4990ca80e8-ca-trust-extracted\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:12:09.788975 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.788812 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxlrq\" (UniqueName: \"kubernetes.io/projected/22ace845-d4cb-4788-b099-8b4990ca80e8-kube-api-access-cxlrq\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:12:09.788975 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:09.788826 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/22ace845-d4cb-4788-b099-8b4990ca80e8-registry-certificates\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:12:10.520685 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:10.520650 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" event={"ID":"22ace845-d4cb-4788-b099-8b4990ca80e8","Type":"ContainerDied","Data":"54dba4b288e1038af5cd817bbd5bc47f356b9c060e449d2b5f8784b0a634d917"} Jul 28 17:12:10.521198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:10.520706 2562 scope.go:117] "RemoveContainer" containerID="114f1c1e2a0ecccdf7430a4decf67d741c6092f4e8aefe4c0a598af40aa035ef" Jul 28 17:12:10.521198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:10.520663 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b54dc8599-qn4bf" Jul 28 17:12:10.541531 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:10.541499 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b54dc8599-qn4bf"] Jul 28 17:12:10.545239 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:10.545213 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7b54dc8599-qn4bf"] Jul 28 17:12:10.929790 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:10.929703 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:12:11.945370 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:11.945332 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22ace845-d4cb-4788-b099-8b4990ca80e8" path="/var/lib/kubelet/pods/22ace845-d4cb-4788-b099-8b4990ca80e8/volumes" Jul 28 17:12:23.996806 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:23.996775 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:12:24.000662 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:24.000634 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5c779bb47b-8tx5c" Jul 28 17:12:45.929326 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:45.929290 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:12:45.946225 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:45.946201 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:12:46.672599 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:12:46.672568 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:03.958054 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:03.958015 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:13:03.958587 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:03.958285 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="prometheus" containerID="cri-o://6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492" gracePeriod=600 Jul 28 17:13:03.958587 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:03.958333 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="kube-rbac-proxy" containerID="cri-o://b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0" gracePeriod=600 Jul 28 17:13:03.958587 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:03.958372 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="kube-rbac-proxy-thanos" containerID="cri-o://711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0" gracePeriod=600 Jul 28 17:13:03.958587 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:03.958375 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="thanos-sidecar" containerID="cri-o://94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450" gracePeriod=600 Jul 28 17:13:03.958587 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:03.958432 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="config-reloader" containerID="cri-o://421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad" gracePeriod=600 Jul 28 17:13:03.958587 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:03.958503 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="kube-rbac-proxy-web" containerID="cri-o://98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96" gracePeriod=600 Jul 28 17:13:04.207210 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.207181 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.365161 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365122 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-web-config\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365161 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365166 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cdd06f73-9108-4ae6-88db-6d661dc3be64-config-out\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365368 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365185 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-trusted-ca-bundle\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365368 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365212 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-tls\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365445 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365414 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwxlb\" (UniqueName: \"kubernetes.io/projected/cdd06f73-9108-4ae6-88db-6d661dc3be64-kube-api-access-vwxlb\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365492 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365463 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-metrics-client-ca\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365545 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365492 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-kubelet-serving-ca-bundle\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365545 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365525 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-metrics-client-certs\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365657 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365549 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-2\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365657 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365571 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-1\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365657 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365568 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:13:04.365657 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365599 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-0\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365657 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365626 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-grpc-tls\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365965 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365662 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365965 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365719 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cdd06f73-9108-4ae6-88db-6d661dc3be64-tls-assets\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365965 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365746 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-thanos-prometheus-http-client-file\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365965 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365778 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-kube-rbac-proxy\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365965 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365815 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-serving-certs-ca-bundle\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365965 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365825 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:13:04.365965 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365853 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365965 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365896 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-db\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.365965 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365923 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-config\") pod \"cdd06f73-9108-4ae6-88db-6d661dc3be64\" (UID: \"cdd06f73-9108-4ae6-88db-6d661dc3be64\") " Jul 28 17:13:04.366406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.366219 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.366406 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.366239 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.368649 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.365868 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:13:04.368649 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.368403 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-config" (OuterVolumeSpecName: "config") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:13:04.368649 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.366137 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:13:04.368649 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.367197 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:13:04.368649 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.367447 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:13:04.368649 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.367833 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:13:04.368649 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.368080 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdd06f73-9108-4ae6-88db-6d661dc3be64-kube-api-access-vwxlb" (OuterVolumeSpecName: "kube-api-access-vwxlb") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "kube-api-access-vwxlb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:13:04.368649 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.368339 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdd06f73-9108-4ae6-88db-6d661dc3be64-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:13:04.368649 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.368368 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:13:04.368649 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.368374 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:13:04.369309 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.369242 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdd06f73-9108-4ae6-88db-6d661dc3be64-config-out" (OuterVolumeSpecName: "config-out") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:13:04.369373 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.369324 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:13:04.370319 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.370285 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:13:04.370430 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.370406 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:13:04.370512 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.370491 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:13:04.370857 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.370840 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:13:04.371149 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.371130 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:13:04.380374 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.380347 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-web-config" (OuterVolumeSpecName: "web-config") pod "cdd06f73-9108-4ae6-88db-6d661dc3be64" (UID: "cdd06f73-9108-4ae6-88db-6d661dc3be64"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:13:04.467100 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467061 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467100 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467095 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwxlb\" (UniqueName: \"kubernetes.io/projected/cdd06f73-9108-4ae6-88db-6d661dc3be64-kube-api-access-vwxlb\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467100 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467105 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-metrics-client-ca\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467117 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467127 2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-metrics-client-certs\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467137 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467145 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467154 2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-grpc-tls\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467162 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467171 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cdd06f73-9108-4ae6-88db-6d661dc3be64-tls-assets\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467179 2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467187 2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-kube-rbac-proxy\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467195 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd06f73-9108-4ae6-88db-6d661dc3be64-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467204 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467212 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cdd06f73-9108-4ae6-88db-6d661dc3be64-prometheus-k8s-db\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467220 2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-config\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467228 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cdd06f73-9108-4ae6-88db-6d661dc3be64-web-config\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.467313 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.467236 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cdd06f73-9108-4ae6-88db-6d661dc3be64-config-out\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:13:04.730701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.730671 2562 generic.go:356] "Generic (PLEG): container finished" podID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerID="711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0" exitCode=0 Jul 28 17:13:04.730701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.730695 2562 generic.go:356] "Generic (PLEG): container finished" podID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerID="b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0" exitCode=0 Jul 28 17:13:04.730701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.730703 2562 generic.go:356] "Generic (PLEG): container finished" podID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerID="98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96" exitCode=0 Jul 28 17:13:04.730701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.730708 2562 generic.go:356] "Generic (PLEG): container finished" podID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerID="94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450" exitCode=0 Jul 28 17:13:04.731004 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.730713 2562 generic.go:356] "Generic (PLEG): container finished" podID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerID="421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad" exitCode=0 Jul 28 17:13:04.731004 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.730719 2562 generic.go:356] "Generic (PLEG): container finished" podID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerID="6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492" exitCode=0 Jul 28 17:13:04.731004 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.730746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdd06f73-9108-4ae6-88db-6d661dc3be64","Type":"ContainerDied","Data":"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0"} Jul 28 17:13:04.731004 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.730755 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.731004 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.730845 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdd06f73-9108-4ae6-88db-6d661dc3be64","Type":"ContainerDied","Data":"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0"} Jul 28 17:13:04.731004 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.730889 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdd06f73-9108-4ae6-88db-6d661dc3be64","Type":"ContainerDied","Data":"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96"} Jul 28 17:13:04.731004 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.730895 2562 scope.go:117] "RemoveContainer" containerID="711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0" Jul 28 17:13:04.731004 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.730921 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdd06f73-9108-4ae6-88db-6d661dc3be64","Type":"ContainerDied","Data":"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450"} Jul 28 17:13:04.731004 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.730946 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdd06f73-9108-4ae6-88db-6d661dc3be64","Type":"ContainerDied","Data":"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad"} Jul 28 17:13:04.731004 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.730970 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdd06f73-9108-4ae6-88db-6d661dc3be64","Type":"ContainerDied","Data":"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492"} Jul 28 17:13:04.731004 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.730994 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdd06f73-9108-4ae6-88db-6d661dc3be64","Type":"ContainerDied","Data":"427a0a2e706d69ec45aeec19c550694f0c81a8d07b395848e8ef3c0b90eef575"} Jul 28 17:13:04.739071 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.739051 2562 scope.go:117] "RemoveContainer" containerID="b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0" Jul 28 17:13:04.745907 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.745870 2562 scope.go:117] "RemoveContainer" containerID="98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96" Jul 28 17:13:04.752432 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.752415 2562 scope.go:117] "RemoveContainer" containerID="94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450" Jul 28 17:13:04.756417 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.756365 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:13:04.761341 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.761317 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:13:04.761784 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.761767 2562 scope.go:117] "RemoveContainer" containerID="421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad" Jul 28 17:13:04.768907 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.768871 2562 scope.go:117] "RemoveContainer" containerID="6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492" Jul 28 17:13:04.775825 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.775809 2562 scope.go:117] "RemoveContainer" containerID="6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a" Jul 28 17:13:04.782730 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.782714 2562 scope.go:117] "RemoveContainer" containerID="711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0" Jul 28 17:13:04.783038 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:13:04.783014 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0\": container with ID starting with 711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0 not found: ID does not exist" containerID="711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0" Jul 28 17:13:04.783107 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.783048 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0"} err="failed to get container status \"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0\": rpc error: code = NotFound desc = could not find container \"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0\": container with ID starting with 711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0 not found: ID does not exist" Jul 28 17:13:04.783107 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.783081 2562 scope.go:117] "RemoveContainer" containerID="b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0" Jul 28 17:13:04.783312 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:13:04.783296 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0\": container with ID starting with b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0 not found: ID does not exist" containerID="b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0" Jul 28 17:13:04.783352 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.783318 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0"} err="failed to get container status \"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0\": rpc error: code = NotFound desc = could not find container \"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0\": container with ID starting with b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0 not found: ID does not exist" Jul 28 17:13:04.783352 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.783334 2562 scope.go:117] "RemoveContainer" containerID="98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96" Jul 28 17:13:04.783561 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:13:04.783546 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96\": container with ID starting with 98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96 not found: ID does not exist" containerID="98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96" Jul 28 17:13:04.783599 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.783564 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96"} err="failed to get container status \"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96\": rpc error: code = NotFound desc = could not find container \"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96\": container with ID starting with 98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96 not found: ID does not exist" Jul 28 17:13:04.783599 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.783578 2562 scope.go:117] "RemoveContainer" containerID="94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450" Jul 28 17:13:04.783792 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:13:04.783775 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450\": container with ID starting with 94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450 not found: ID does not exist" containerID="94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450" Jul 28 17:13:04.783842 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.783795 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450"} err="failed to get container status \"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450\": rpc error: code = NotFound desc = could not find container \"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450\": container with ID starting with 94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450 not found: ID does not exist" Jul 28 17:13:04.783842 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.783809 2562 scope.go:117] "RemoveContainer" containerID="421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad" Jul 28 17:13:04.784026 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:13:04.784011 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad\": container with ID starting with 421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad not found: ID does not exist" containerID="421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad" Jul 28 17:13:04.784066 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.784030 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad"} err="failed to get container status \"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad\": rpc error: code = NotFound desc = could not find container \"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad\": container with ID starting with 421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad not found: ID does not exist" Jul 28 17:13:04.784066 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.784043 2562 scope.go:117] "RemoveContainer" containerID="6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492" Jul 28 17:13:04.784232 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:13:04.784218 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492\": container with ID starting with 6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492 not found: ID does not exist" containerID="6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492" Jul 28 17:13:04.784270 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.784234 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492"} err="failed to get container status \"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492\": rpc error: code = NotFound desc = could not find container \"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492\": container with ID starting with 6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492 not found: ID does not exist" Jul 28 17:13:04.784270 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.784245 2562 scope.go:117] "RemoveContainer" containerID="6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a" Jul 28 17:13:04.784451 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:13:04.784435 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a\": container with ID starting with 6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a not found: ID does not exist" containerID="6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a" Jul 28 17:13:04.784495 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.784453 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a"} err="failed to get container status \"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a\": rpc error: code = NotFound desc = could not find container \"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a\": container with ID starting with 6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a not found: ID does not exist" Jul 28 17:13:04.784495 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.784465 2562 scope.go:117] "RemoveContainer" containerID="711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0" Jul 28 17:13:04.784644 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.784630 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0"} err="failed to get container status \"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0\": rpc error: code = NotFound desc = could not find container \"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0\": container with ID starting with 711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0 not found: ID does not exist" Jul 28 17:13:04.784688 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.784644 2562 scope.go:117] "RemoveContainer" containerID="b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0" Jul 28 17:13:04.784844 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.784828 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0"} err="failed to get container status \"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0\": rpc error: code = NotFound desc = could not find container \"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0\": container with ID starting with b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0 not found: ID does not exist" Jul 28 17:13:04.784907 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.784844 2562 scope.go:117] "RemoveContainer" containerID="98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96" Jul 28 17:13:04.785079 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.785061 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96"} err="failed to get container status \"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96\": rpc error: code = NotFound desc = could not find container \"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96\": container with ID starting with 98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96 not found: ID does not exist" Jul 28 17:13:04.785130 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.785082 2562 scope.go:117] "RemoveContainer" containerID="94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450" Jul 28 17:13:04.785295 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.785280 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450"} err="failed to get container status \"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450\": rpc error: code = NotFound desc = could not find container \"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450\": container with ID starting with 94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450 not found: ID does not exist" Jul 28 17:13:04.785334 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.785296 2562 scope.go:117] "RemoveContainer" containerID="421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad" Jul 28 17:13:04.785497 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.785482 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad"} err="failed to get container status \"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad\": rpc error: code = NotFound desc = could not find container \"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad\": container with ID starting with 421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad not found: ID does not exist" Jul 28 17:13:04.785539 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.785497 2562 scope.go:117] "RemoveContainer" containerID="6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492" Jul 28 17:13:04.785694 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.785679 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492"} err="failed to get container status \"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492\": rpc error: code = NotFound desc = could not find container \"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492\": container with ID starting with 6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492 not found: ID does not exist" Jul 28 17:13:04.785753 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.785694 2562 scope.go:117] "RemoveContainer" containerID="6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a" Jul 28 17:13:04.785925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.785896 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a"} err="failed to get container status \"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a\": rpc error: code = NotFound desc = could not find container \"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a\": container with ID starting with 6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a not found: ID does not exist" Jul 28 17:13:04.785925 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.785910 2562 scope.go:117] "RemoveContainer" containerID="711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0" Jul 28 17:13:04.786114 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.786099 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0"} err="failed to get container status \"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0\": rpc error: code = NotFound desc = could not find container \"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0\": container with ID starting with 711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0 not found: ID does not exist" Jul 28 17:13:04.786157 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.786114 2562 scope.go:117] "RemoveContainer" containerID="b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0" Jul 28 17:13:04.786300 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.786284 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0"} err="failed to get container status \"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0\": rpc error: code = NotFound desc = could not find container \"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0\": container with ID starting with b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0 not found: ID does not exist" Jul 28 17:13:04.786346 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.786300 2562 scope.go:117] "RemoveContainer" containerID="98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96" Jul 28 17:13:04.786497 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.786480 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96"} err="failed to get container status \"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96\": rpc error: code = NotFound desc = could not find container \"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96\": container with ID starting with 98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96 not found: ID does not exist" Jul 28 17:13:04.786497 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.786496 2562 scope.go:117] "RemoveContainer" containerID="94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450" Jul 28 17:13:04.786667 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.786653 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450"} err="failed to get container status \"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450\": rpc error: code = NotFound desc = could not find container \"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450\": container with ID starting with 94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450 not found: ID does not exist" Jul 28 17:13:04.786708 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.786667 2562 scope.go:117] "RemoveContainer" containerID="421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad" Jul 28 17:13:04.786833 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.786819 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad"} err="failed to get container status \"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad\": rpc error: code = NotFound desc = could not find container \"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad\": container with ID starting with 421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad not found: ID does not exist" Jul 28 17:13:04.786833 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.786832 2562 scope.go:117] "RemoveContainer" containerID="6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492" Jul 28 17:13:04.787091 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.787062 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492"} err="failed to get container status \"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492\": rpc error: code = NotFound desc = could not find container \"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492\": container with ID starting with 6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492 not found: ID does not exist" Jul 28 17:13:04.787091 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.787091 2562 scope.go:117] "RemoveContainer" containerID="6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a" Jul 28 17:13:04.787350 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.787329 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a"} err="failed to get container status \"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a\": rpc error: code = NotFound desc = could not find container \"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a\": container with ID starting with 6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a not found: ID does not exist" Jul 28 17:13:04.787431 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.787352 2562 scope.go:117] "RemoveContainer" containerID="711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0" Jul 28 17:13:04.787807 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.787779 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0"} err="failed to get container status \"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0\": rpc error: code = NotFound desc = could not find container \"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0\": container with ID starting with 711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0 not found: ID does not exist" Jul 28 17:13:04.787971 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.787809 2562 scope.go:117] "RemoveContainer" containerID="b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0" Jul 28 17:13:04.788219 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.788184 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0"} err="failed to get container status \"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0\": rpc error: code = NotFound desc = could not find container \"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0\": container with ID starting with b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0 not found: ID does not exist" Jul 28 17:13:04.788219 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.788213 2562 scope.go:117] "RemoveContainer" containerID="98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96" Jul 28 17:13:04.788491 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.788464 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96"} err="failed to get container status \"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96\": rpc error: code = NotFound desc = could not find container \"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96\": container with ID starting with 98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96 not found: ID does not exist" Jul 28 17:13:04.788491 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.788490 2562 scope.go:117] "RemoveContainer" containerID="94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450" Jul 28 17:13:04.788752 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.788734 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450"} err="failed to get container status \"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450\": rpc error: code = NotFound desc = could not find container \"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450\": container with ID starting with 94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450 not found: ID does not exist" Jul 28 17:13:04.788821 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.788753 2562 scope.go:117] "RemoveContainer" containerID="421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad" Jul 28 17:13:04.789135 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789107 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad"} err="failed to get container status \"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad\": rpc error: code = NotFound desc = could not find container \"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad\": container with ID starting with 421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad not found: ID does not exist" Jul 28 17:13:04.789198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789138 2562 scope.go:117] "RemoveContainer" containerID="6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492" Jul 28 17:13:04.789373 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789353 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492"} err="failed to get container status \"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492\": rpc error: code = NotFound desc = could not find container \"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492\": container with ID starting with 6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492 not found: ID does not exist" Jul 28 17:13:04.789433 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789374 2562 scope.go:117] "RemoveContainer" containerID="6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a" Jul 28 17:13:04.789522 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789499 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:13:04.789613 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789529 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a"} err="failed to get container status \"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a\": rpc error: code = NotFound desc = could not find container \"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a\": container with ID starting with 6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a not found: ID does not exist" Jul 28 17:13:04.789613 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789541 2562 scope.go:117] "RemoveContainer" containerID="711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0" Jul 28 17:13:04.789756 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789737 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0"} err="failed to get container status \"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0\": rpc error: code = NotFound desc = could not find container \"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0\": container with ID starting with 711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0 not found: ID does not exist" Jul 28 17:13:04.789798 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789755 2562 scope.go:117] "RemoveContainer" containerID="b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0" Jul 28 17:13:04.789798 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789788 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22ace845-d4cb-4788-b099-8b4990ca80e8" containerName="registry" Jul 28 17:13:04.789888 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789799 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="22ace845-d4cb-4788-b099-8b4990ca80e8" containerName="registry" Jul 28 17:13:04.789888 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789809 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="config-reloader" Jul 28 17:13:04.789888 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789814 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="config-reloader" Jul 28 17:13:04.789888 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789821 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="kube-rbac-proxy-thanos" Jul 28 17:13:04.789888 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789829 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="kube-rbac-proxy-thanos" Jul 28 17:13:04.789888 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789838 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="thanos-sidecar" Jul 28 17:13:04.789888 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789843 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="thanos-sidecar" Jul 28 17:13:04.789888 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789849 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="init-config-reloader" Jul 28 17:13:04.789888 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789854 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="init-config-reloader" Jul 28 17:13:04.790198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789863 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="kube-rbac-proxy" Jul 28 17:13:04.790198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789903 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="kube-rbac-proxy" Jul 28 17:13:04.790198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789912 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="prometheus" Jul 28 17:13:04.790198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789917 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="prometheus" Jul 28 17:13:04.790198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789931 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="kube-rbac-proxy-web" Jul 28 17:13:04.790198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789936 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="kube-rbac-proxy-web" Jul 28 17:13:04.790198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789978 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0"} err="failed to get container status \"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0\": rpc error: code = NotFound desc = could not find container \"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0\": container with ID starting with b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0 not found: ID does not exist" Jul 28 17:13:04.790198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789993 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="config-reloader" Jul 28 17:13:04.790198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.789997 2562 scope.go:117] "RemoveContainer" containerID="98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96" Jul 28 17:13:04.790198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.790001 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="kube-rbac-proxy-thanos" Jul 28 17:13:04.790198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.790008 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="prometheus" Jul 28 17:13:04.790198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.790014 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="thanos-sidecar" Jul 28 17:13:04.790198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.790020 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="22ace845-d4cb-4788-b099-8b4990ca80e8" containerName="registry" Jul 28 17:13:04.790198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.790035 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="kube-rbac-proxy" Jul 28 17:13:04.790198 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.790044 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" containerName="kube-rbac-proxy-web" Jul 28 17:13:04.790662 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.790215 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96"} err="failed to get container status \"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96\": rpc error: code = NotFound desc = could not find container \"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96\": container with ID starting with 98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96 not found: ID does not exist" Jul 28 17:13:04.790662 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.790235 2562 scope.go:117] "RemoveContainer" containerID="94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450" Jul 28 17:13:04.790662 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.790457 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450"} err="failed to get container status \"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450\": rpc error: code = NotFound desc = could not find container \"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450\": container with ID starting with 94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450 not found: ID does not exist" Jul 28 17:13:04.790662 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.790478 2562 scope.go:117] "RemoveContainer" containerID="421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad" Jul 28 17:13:04.790799 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.790712 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad"} err="failed to get container status \"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad\": rpc error: code = NotFound desc = could not find container \"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad\": container with ID starting with 421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad not found: ID does not exist" Jul 28 17:13:04.790799 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.790728 2562 scope.go:117] "RemoveContainer" containerID="6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492" Jul 28 17:13:04.790965 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.790944 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492"} err="failed to get container status \"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492\": rpc error: code = NotFound desc = could not find container \"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492\": container with ID starting with 6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492 not found: ID does not exist" Jul 28 17:13:04.791042 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.790967 2562 scope.go:117] "RemoveContainer" containerID="6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a" Jul 28 17:13:04.791217 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.791194 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a"} err="failed to get container status \"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a\": rpc error: code = NotFound desc = could not find container \"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a\": container with ID starting with 6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a not found: ID does not exist" Jul 28 17:13:04.791263 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.791219 2562 scope.go:117] "RemoveContainer" containerID="711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0" Jul 28 17:13:04.791420 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.791402 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0"} err="failed to get container status \"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0\": rpc error: code = NotFound desc = could not find container \"711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0\": container with ID starting with 711dd00a1a4d7204a78727577f3e94e5161e569ad02ab99d5a6087028d24a6c0 not found: ID does not exist" Jul 28 17:13:04.791480 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.791421 2562 scope.go:117] "RemoveContainer" containerID="b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0" Jul 28 17:13:04.791624 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.791606 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0"} err="failed to get container status \"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0\": rpc error: code = NotFound desc = could not find container \"b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0\": container with ID starting with b2a74b2c5d9bd89672e157117c6414f32e6548c16adcdc22867a0dddbf221db0 not found: ID does not exist" Jul 28 17:13:04.791691 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.791626 2562 scope.go:117] "RemoveContainer" containerID="98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96" Jul 28 17:13:04.791824 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.791804 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96"} err="failed to get container status \"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96\": rpc error: code = NotFound desc = could not find container \"98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96\": container with ID starting with 98ce50029d418cb3257f5df961cd38e5e0dc3438b3cd2f412eb38704342c2b96 not found: ID does not exist" Jul 28 17:13:04.792038 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.791825 2562 scope.go:117] "RemoveContainer" containerID="94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450" Jul 28 17:13:04.792084 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.792052 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450"} err="failed to get container status \"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450\": rpc error: code = NotFound desc = could not find container \"94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450\": container with ID starting with 94ba5529fe132c116f1a6174704abc7e7996ecaa9a854a2d9d327ab4dc119450 not found: ID does not exist" Jul 28 17:13:04.792084 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.792066 2562 scope.go:117] "RemoveContainer" containerID="421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad" Jul 28 17:13:04.792297 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.792281 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad"} err="failed to get container status \"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad\": rpc error: code = NotFound desc = could not find container \"421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad\": container with ID starting with 421bae1629f98ffa06cf3296a81ad92770d4462d9085a4a87b493de7413736ad not found: ID does not exist" Jul 28 17:13:04.792353 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.792297 2562 scope.go:117] "RemoveContainer" containerID="6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492" Jul 28 17:13:04.792523 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.792507 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492"} err="failed to get container status \"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492\": rpc error: code = NotFound desc = could not find container \"6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492\": container with ID starting with 6b38a486055d9ac1956d951d289adb7a15cfc460188a34ae9db7a8156cbb5492 not found: ID does not exist" Jul 28 17:13:04.792566 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.792524 2562 scope.go:117] "RemoveContainer" containerID="6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a" Jul 28 17:13:04.792728 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.792710 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a"} err="failed to get container status \"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a\": rpc error: code = NotFound desc = could not find container \"6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a\": container with ID starting with 6c07eed3043b37dab92810d92873123ad17d8096a5d0ac6d144f545d731d0e3a not found: ID does not exist" Jul 28 17:13:04.794601 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.794581 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.797842 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.797816 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 28 17:13:04.797967 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.797949 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 28 17:13:04.798017 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.797988 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 28 17:13:04.798017 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.797949 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 28 17:13:04.798127 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.798090 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 28 17:13:04.798307 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.798268 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 28 17:13:04.798459 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.798379 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 28 17:13:04.798459 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.798389 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 28 17:13:04.798595 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.798480 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-cnkp5\"" Jul 28 17:13:04.798595 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.798515 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7lvu1lj77bi9d\"" Jul 28 17:13:04.798688 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.798605 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 28 17:13:04.798688 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.798623 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 28 17:13:04.799475 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.799459 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 28 17:13:04.799837 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.799817 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 28 17:13:04.801746 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.801727 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 28 17:13:04.804195 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.804178 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 28 17:13:04.809827 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.809791 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:13:04.971384 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971352 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971384 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971392 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-web-config\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971416 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-config-out\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971444 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971466 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971483 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971510 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971549 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971588 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971619 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971634 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971655 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971681 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-config\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971698 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dxf6\" (UniqueName: \"kubernetes.io/projected/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-kube-api-access-4dxf6\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971718 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971744 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971778 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.971808 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971802 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.972285 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971843 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:04.972285 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:04.971859 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073227 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073194 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073227 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073228 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073454 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073252 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073454 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073277 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-web-config\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073454 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073414 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-config-out\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073454 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073444 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073633 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073472 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073633 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073494 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073633 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073633 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073549 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073633 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073597 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073907 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073653 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073907 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073680 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073907 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073720 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073907 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073753 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-config\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073907 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073778 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dxf6\" (UniqueName: \"kubernetes.io/projected/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-kube-api-access-4dxf6\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073907 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073907 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073847 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.073907 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073893 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.074345 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.073917 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.074345 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.074047 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.075626 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.074502 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.075626 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.074962 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.075626 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.075115 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.075626 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.075452 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.078766 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.076173 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-config-out\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.078766 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.076382 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.078766 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.076708 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.078766 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.076842 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.078766 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.076941 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-web-config\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.078766 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.077237 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.078766 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.077437 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.078766 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.077586 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.078766 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.078670 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.079312 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.079204 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.079460 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.079431 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.079855 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.079831 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-config\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.079855 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.079851 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.080237 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.080218 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.091382 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.091355 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dxf6\" (UniqueName: \"kubernetes.io/projected/fab3ebaa-7846-41e4-ab5a-5390fbad1cec-kube-api-access-4dxf6\") pod \"prometheus-k8s-0\" (UID: \"fab3ebaa-7846-41e4-ab5a-5390fbad1cec\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.106205 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.106178 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:13:05.251236 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.251131 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:13:05.254286 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:13:05.254249 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfab3ebaa_7846_41e4_ab5a_5390fbad1cec.slice/crio-90d0dcc28b7014c156a56603fee0affcdf03bd3121a53779d2d3d8e881c9ed75 WatchSource:0}: Error finding container 90d0dcc28b7014c156a56603fee0affcdf03bd3121a53779d2d3d8e881c9ed75: Status 404 returned error can't find the container with id 90d0dcc28b7014c156a56603fee0affcdf03bd3121a53779d2d3d8e881c9ed75 Jul 28 17:13:05.736904 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.736703 2562 generic.go:356] "Generic (PLEG): container finished" podID="fab3ebaa-7846-41e4-ab5a-5390fbad1cec" containerID="bd338800b7d87c71733c21390d50a7593970307f0bca3ea6ef6afb7b074ab026" exitCode=0 Jul 28 17:13:05.740919 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.737142 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fab3ebaa-7846-41e4-ab5a-5390fbad1cec","Type":"ContainerDied","Data":"bd338800b7d87c71733c21390d50a7593970307f0bca3ea6ef6afb7b074ab026"} Jul 28 17:13:05.740919 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.737289 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fab3ebaa-7846-41e4-ab5a-5390fbad1cec","Type":"ContainerStarted","Data":"90d0dcc28b7014c156a56603fee0affcdf03bd3121a53779d2d3d8e881c9ed75"} Jul 28 17:13:05.952579 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:05.952363 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdd06f73-9108-4ae6-88db-6d661dc3be64" path="/var/lib/kubelet/pods/cdd06f73-9108-4ae6-88db-6d661dc3be64/volumes" Jul 28 17:13:06.749702 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:06.749665 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fab3ebaa-7846-41e4-ab5a-5390fbad1cec","Type":"ContainerStarted","Data":"f758116357ee12b382515f0a762891fe41cea8c2d95217b377ea8be13a47ecbe"} Jul 28 17:13:06.750220 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:06.749741 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fab3ebaa-7846-41e4-ab5a-5390fbad1cec","Type":"ContainerStarted","Data":"c84073cdff552426bc2a97e2c6527e8f8046c0bd13152b8a911191411ac1d05b"} Jul 28 17:13:06.750220 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:06.749781 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fab3ebaa-7846-41e4-ab5a-5390fbad1cec","Type":"ContainerStarted","Data":"ce82941b0b2921c31c73c43129f4dc1ae45ffa1a5801df72950dbb7e6f9f5f58"} Jul 28 17:13:06.750220 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:06.749820 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fab3ebaa-7846-41e4-ab5a-5390fbad1cec","Type":"ContainerStarted","Data":"945d35e0610313c0a8ed621fa0a961ff5c732f977fd3f3a91959e4c464b9f7e9"} Jul 28 17:13:06.750220 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:06.749866 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fab3ebaa-7846-41e4-ab5a-5390fbad1cec","Type":"ContainerStarted","Data":"fb9319330c0cbc5c3732b66134f7a72ae56d30dbb7b47181c090049ccf1439d3"} Jul 28 17:13:06.750220 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:06.749923 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fab3ebaa-7846-41e4-ab5a-5390fbad1cec","Type":"ContainerStarted","Data":"3625144452fcedd638de706f313bff6666c70db106ff0fba12261d4211d00d80"} Jul 28 17:13:06.785246 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:06.785184 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.767734096 podStartE2EDuration="2.785165407s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.431311ms" isStatefulPod=false podCreationTimestamp="2026-07-28 17:13:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:06.783183478 +0000 UTC m=+169.383635827" watchObservedRunningTime="2026-07-28 17:13:06.785165407 +0000 UTC m=+169.385617777" Jul 28 17:13:10.107246 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:13:10.107150 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:14:05.106975 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:14:05.106915 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:14:05.122643 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:14:05.122613 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:14:05.998661 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:14:05.998634 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:15:17.820859 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:15:17.820827 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 28 17:18:58.779652 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:58.779619 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-bwcxm"] Jul 28 17:18:58.783952 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:58.783919 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-bwcxm" Jul 28 17:18:58.787556 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:58.787516 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 28 17:18:58.787556 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:58.787537 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-lcbwl\"" Jul 28 17:18:58.787744 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:58.787537 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 28 17:18:58.789325 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:58.788938 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 28 17:18:58.791372 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:58.791350 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-bwcxm"] Jul 28 17:18:58.839917 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:58.839858 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwlf7\" (UniqueName: \"kubernetes.io/projected/54358545-3801-44f1-952e-5aba029ea28b-kube-api-access-gwlf7\") pod \"s3-init-bwcxm\" (UID: \"54358545-3801-44f1-952e-5aba029ea28b\") " pod="kserve/s3-init-bwcxm" Jul 28 17:18:58.839917 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:58.839915 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54358545-3801-44f1-952e-5aba029ea28b-model-cache\") pod \"s3-init-bwcxm\" (UID: \"54358545-3801-44f1-952e-5aba029ea28b\") " pod="kserve/s3-init-bwcxm" Jul 28 17:18:58.940469 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:58.940434 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwlf7\" (UniqueName: \"kubernetes.io/projected/54358545-3801-44f1-952e-5aba029ea28b-kube-api-access-gwlf7\") pod \"s3-init-bwcxm\" (UID: \"54358545-3801-44f1-952e-5aba029ea28b\") " pod="kserve/s3-init-bwcxm" Jul 28 17:18:58.940643 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:58.940477 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54358545-3801-44f1-952e-5aba029ea28b-model-cache\") pod \"s3-init-bwcxm\" (UID: \"54358545-3801-44f1-952e-5aba029ea28b\") " pod="kserve/s3-init-bwcxm" Jul 28 17:18:58.940833 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:58.940817 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54358545-3801-44f1-952e-5aba029ea28b-model-cache\") pod \"s3-init-bwcxm\" (UID: \"54358545-3801-44f1-952e-5aba029ea28b\") " pod="kserve/s3-init-bwcxm" Jul 28 17:18:58.949539 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:58.949510 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwlf7\" (UniqueName: \"kubernetes.io/projected/54358545-3801-44f1-952e-5aba029ea28b-kube-api-access-gwlf7\") pod \"s3-init-bwcxm\" (UID: \"54358545-3801-44f1-952e-5aba029ea28b\") " pod="kserve/s3-init-bwcxm" Jul 28 17:18:59.106752 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:59.106670 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-bwcxm" Jul 28 17:18:59.232143 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:59.232101 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-bwcxm"] Jul 28 17:18:59.234636 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:18:59.234612 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod54358545_3801_44f1_952e_5aba029ea28b.slice/crio-be6187064b1739118e8e76127c83a13f9d47db836289b62942c8491c603a22e1 WatchSource:0}: Error finding container be6187064b1739118e8e76127c83a13f9d47db836289b62942c8491c603a22e1: Status 404 returned error can't find the container with id be6187064b1739118e8e76127c83a13f9d47db836289b62942c8491c603a22e1 Jul 28 17:18:59.237111 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:18:59.237096 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:19:00.110947 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:19:00.110856 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-bwcxm" event={"ID":"54358545-3801-44f1-952e-5aba029ea28b","Type":"ContainerStarted","Data":"be6187064b1739118e8e76127c83a13f9d47db836289b62942c8491c603a22e1"} Jul 28 17:19:03.124627 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:19:03.124586 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-bwcxm" event={"ID":"54358545-3801-44f1-952e-5aba029ea28b","Type":"ContainerStarted","Data":"a0a6f2164f9d6d1c76dee24abb83935e041c1f04fb374f3666ad76f67edc82a3"} Jul 28 17:19:12.163001 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:19:12.162916 2562 generic.go:356] "Generic (PLEG): container finished" podID="54358545-3801-44f1-952e-5aba029ea28b" containerID="a0a6f2164f9d6d1c76dee24abb83935e041c1f04fb374f3666ad76f67edc82a3" exitCode=0 Jul 28 17:19:12.163001 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:19:12.162959 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-bwcxm" event={"ID":"54358545-3801-44f1-952e-5aba029ea28b","Type":"ContainerDied","Data":"a0a6f2164f9d6d1c76dee24abb83935e041c1f04fb374f3666ad76f67edc82a3"} Jul 28 17:19:17.187012 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:19:17.186966 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-bwcxm" event={"ID":"54358545-3801-44f1-952e-5aba029ea28b","Type":"ContainerStarted","Data":"fcc93c134b1ab2da2ef663f89a61f9952685444af570a24f3c699a338cb9be0a"} Jul 28 17:19:17.208595 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:19:17.208541 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-bwcxm" podStartSLOduration=2.324403863 podStartE2EDuration="19.208526726s" totalImagesPullingTime="7.647394626s" totalInitContainerRuntime="9.236728237s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:19:17.206582762 +0000 UTC m=+539.807035107" watchObservedRunningTime="2026-07-28 17:19:17.208526726 +0000 UTC m=+539.808979073" Jul 28 17:20:22.439320 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:22.439280 2562 generic.go:356] "Generic (PLEG): container finished" podID="54358545-3801-44f1-952e-5aba029ea28b" containerID="fcc93c134b1ab2da2ef663f89a61f9952685444af570a24f3c699a338cb9be0a" exitCode=0 Jul 28 17:20:22.439764 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:22.439315 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-bwcxm" event={"ID":"54358545-3801-44f1-952e-5aba029ea28b","Type":"ContainerDied","Data":"fcc93c134b1ab2da2ef663f89a61f9952685444af570a24f3c699a338cb9be0a"} Jul 28 17:20:23.574381 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:23.574357 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-bwcxm" Jul 28 17:20:23.666537 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:23.666496 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwlf7\" (UniqueName: \"kubernetes.io/projected/54358545-3801-44f1-952e-5aba029ea28b-kube-api-access-gwlf7\") pod \"54358545-3801-44f1-952e-5aba029ea28b\" (UID: \"54358545-3801-44f1-952e-5aba029ea28b\") " Jul 28 17:20:23.666731 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:23.666554 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54358545-3801-44f1-952e-5aba029ea28b-model-cache\") pod \"54358545-3801-44f1-952e-5aba029ea28b\" (UID: \"54358545-3801-44f1-952e-5aba029ea28b\") " Jul 28 17:20:23.668735 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:23.668701 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54358545-3801-44f1-952e-5aba029ea28b-kube-api-access-gwlf7" (OuterVolumeSpecName: "kube-api-access-gwlf7") pod "54358545-3801-44f1-952e-5aba029ea28b" (UID: "54358545-3801-44f1-952e-5aba029ea28b"). InnerVolumeSpecName "kube-api-access-gwlf7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:20:23.685559 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:23.685514 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54358545-3801-44f1-952e-5aba029ea28b-model-cache" (OuterVolumeSpecName: "model-cache") pod "54358545-3801-44f1-952e-5aba029ea28b" (UID: "54358545-3801-44f1-952e-5aba029ea28b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:20:23.767458 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:23.767419 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwlf7\" (UniqueName: \"kubernetes.io/projected/54358545-3801-44f1-952e-5aba029ea28b-kube-api-access-gwlf7\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:20:23.767458 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:23.767453 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54358545-3801-44f1-952e-5aba029ea28b-model-cache\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:20:24.448803 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:24.448764 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-bwcxm" event={"ID":"54358545-3801-44f1-952e-5aba029ea28b","Type":"ContainerDied","Data":"be6187064b1739118e8e76127c83a13f9d47db836289b62942c8491c603a22e1"} Jul 28 17:20:24.448803 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:24.448803 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be6187064b1739118e8e76127c83a13f9d47db836289b62942c8491c603a22e1" Jul 28 17:20:24.448803 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:24.448807 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-bwcxm" Jul 28 17:20:36.483997 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.483959 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj"] Jul 28 17:20:36.484601 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.484444 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54358545-3801-44f1-952e-5aba029ea28b" containerName="s3-init" Jul 28 17:20:36.484601 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.484474 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="54358545-3801-44f1-952e-5aba029ea28b" containerName="s3-init" Jul 28 17:20:36.484601 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.484487 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54358545-3801-44f1-952e-5aba029ea28b" containerName="download-hf-model" Jul 28 17:20:36.484601 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.484496 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="54358545-3801-44f1-952e-5aba029ea28b" containerName="download-hf-model" Jul 28 17:20:36.484601 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.484568 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="54358545-3801-44f1-952e-5aba029ea28b" containerName="s3-init" Jul 28 17:20:36.487069 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.487049 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.490654 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.490629 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 28 17:20:36.490769 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.490670 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\"" Jul 28 17:20:36.490849 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.490762 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 28 17:20:36.490849 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.490679 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wxvxt\"" Jul 28 17:20:36.490849 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.490781 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\"" Jul 28 17:20:36.491030 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.490939 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 17:20:36.492060 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.492044 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 28 17:20:36.499817 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.499793 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj"] Jul 28 17:20:36.563423 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.563393 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t6zf\" (UniqueName: \"kubernetes.io/projected/c369e90f-8cba-427f-8dac-d0669e473abe-kube-api-access-9t6zf\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.563588 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.563439 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c369e90f-8cba-427f-8dac-d0669e473abe-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.563588 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.563489 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c369e90f-8cba-427f-8dac-d0669e473abe-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.563588 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.563508 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c369e90f-8cba-427f-8dac-d0669e473abe-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.563713 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.563587 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c369e90f-8cba-427f-8dac-d0669e473abe-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.664265 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.664227 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c369e90f-8cba-427f-8dac-d0669e473abe-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.664439 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.664283 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c369e90f-8cba-427f-8dac-d0669e473abe-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.664439 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.664323 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c369e90f-8cba-427f-8dac-d0669e473abe-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.664439 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.664380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t6zf\" (UniqueName: \"kubernetes.io/projected/c369e90f-8cba-427f-8dac-d0669e473abe-kube-api-access-9t6zf\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.664439 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.664426 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c369e90f-8cba-427f-8dac-d0669e473abe-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.664706 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.664681 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c369e90f-8cba-427f-8dac-d0669e473abe-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.665043 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.665020 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c369e90f-8cba-427f-8dac-d0669e473abe-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.665143 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.665123 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c369e90f-8cba-427f-8dac-d0669e473abe-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.666833 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.666812 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c369e90f-8cba-427f-8dac-d0669e473abe-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.676480 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.676456 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t6zf\" (UniqueName: \"kubernetes.io/projected/c369e90f-8cba-427f-8dac-d0669e473abe-kube-api-access-9t6zf\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.720468 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.720433 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z"] Jul 28 17:20:36.722909 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.722892 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:36.726107 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.726077 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\"" Jul 28 17:20:36.726329 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.726313 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\"" Jul 28 17:20:36.733697 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.733671 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z"] Jul 28 17:20:36.765340 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.765241 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:36.765340 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.765289 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:36.765340 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.765335 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:36.765638 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.765405 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbdrk\" (UniqueName: \"kubernetes.io/projected/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kube-api-access-jbdrk\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:36.765638 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.765438 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:36.798547 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.798505 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:20:36.867069 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.866226 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:36.867069 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.866283 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:36.867069 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.866323 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:36.867069 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:20:36.866402 2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-xgboost-graph-predictor-serving-cert: secret "isvc-xgboost-graph-predictor-serving-cert" not found Jul 28 17:20:36.867069 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:20:36.866474 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-proxy-tls podName:ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e nodeName:}" failed. No retries permitted until 2026-07-28 17:20:37.366454546 +0000 UTC m=+619.966906875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-proxy-tls") pod "isvc-xgboost-graph-predictor-555fc69d56-ghf5z" (UID: "ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e") : secret "isvc-xgboost-graph-predictor-serving-cert" not found Jul 28 17:20:36.867069 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.866774 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbdrk\" (UniqueName: \"kubernetes.io/projected/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kube-api-access-jbdrk\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:36.867069 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.866817 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:36.867069 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.866815 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:36.867639 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.867557 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:36.867784 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.867762 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:36.875910 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.875821 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbdrk\" (UniqueName: \"kubernetes.io/projected/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kube-api-access-jbdrk\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:36.959025 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:36.958994 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj"] Jul 28 17:20:36.963354 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:20:36.963325 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc369e90f_8cba_427f_8dac_d0669e473abe.slice/crio-b826609b44d8788d5eefe9b00c62842ff1d5cd120c6dee3277c2cfb63cd50652 WatchSource:0}: Error finding container b826609b44d8788d5eefe9b00c62842ff1d5cd120c6dee3277c2cfb63cd50652: Status 404 returned error can't find the container with id b826609b44d8788d5eefe9b00c62842ff1d5cd120c6dee3277c2cfb63cd50652 Jul 28 17:20:37.370677 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:37.370588 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:37.373076 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:37.373052 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-ghf5z\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:37.499452 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:37.499403 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" event={"ID":"c369e90f-8cba-427f-8dac-d0669e473abe","Type":"ContainerStarted","Data":"c27c752eca262caee3edb387ce7aa7cc548c38908bf651801ed7d04e203428a0"} Jul 28 17:20:37.499861 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:37.499482 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" event={"ID":"c369e90f-8cba-427f-8dac-d0669e473abe","Type":"ContainerStarted","Data":"b826609b44d8788d5eefe9b00c62842ff1d5cd120c6dee3277c2cfb63cd50652"} Jul 28 17:20:37.634122 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:37.634027 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:20:37.768396 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:37.768366 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z"] Jul 28 17:20:37.770029 ip-10-0-139-193 kubenswrapper[2562]: W0728 17:20:37.769998 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebe8c668_bf03_4eb4_9e6d_5b89f3af8f8e.slice/crio-a4480dab4bf70d75f08d8e61edd1552aa5711560e8083e2774bb00969bc91ace WatchSource:0}: Error finding container a4480dab4bf70d75f08d8e61edd1552aa5711560e8083e2774bb00969bc91ace: Status 404 returned error can't find the container with id a4480dab4bf70d75f08d8e61edd1552aa5711560e8083e2774bb00969bc91ace Jul 28 17:20:38.505103 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:38.505059 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" event={"ID":"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e","Type":"ContainerStarted","Data":"e0bbb471572300618f37e8534c89c467fdaf3f99a3dffde553c966e060aa5ade"} Jul 28 17:20:38.505512 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:38.505171 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" event={"ID":"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e","Type":"ContainerStarted","Data":"a4480dab4bf70d75f08d8e61edd1552aa5711560e8083e2774bb00969bc91ace"} Jul 28 17:20:41.518293 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:41.518255 2562 generic.go:356] "Generic (PLEG): container finished" podID="c369e90f-8cba-427f-8dac-d0669e473abe" containerID="c27c752eca262caee3edb387ce7aa7cc548c38908bf651801ed7d04e203428a0" exitCode=0 Jul 28 17:20:41.518726 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:41.518309 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" event={"ID":"c369e90f-8cba-427f-8dac-d0669e473abe","Type":"ContainerDied","Data":"c27c752eca262caee3edb387ce7aa7cc548c38908bf651801ed7d04e203428a0"} Jul 28 17:20:42.530900 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:42.530100 2562 generic.go:356] "Generic (PLEG): container finished" podID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerID="e0bbb471572300618f37e8534c89c467fdaf3f99a3dffde553c966e060aa5ade" exitCode=0 Jul 28 17:20:42.530900 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:20:42.530181 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" event={"ID":"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e","Type":"ContainerDied","Data":"e0bbb471572300618f37e8534c89c467fdaf3f99a3dffde553c966e060aa5ade"} Jul 28 17:21:06.661054 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:06.661020 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" event={"ID":"c369e90f-8cba-427f-8dac-d0669e473abe","Type":"ContainerStarted","Data":"69ce2d993a5f0a5ba1e780add82a2c13804ad129f1004b4af24243715f6a1278"} Jul 28 17:21:07.670590 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:07.670550 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" event={"ID":"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e","Type":"ContainerStarted","Data":"6c034c63470c2fee2b7d536126eab23e67fd821857d1e84dfe7e1d977f27e201"} Jul 28 17:21:10.686413 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:10.686324 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" event={"ID":"c369e90f-8cba-427f-8dac-d0669e473abe","Type":"ContainerStarted","Data":"2fbd4e7ca5c360adc6d81e5df48b585bb4c2ac18129531a45c66a25d979566b5"} Jul 28 17:21:10.686891 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:10.686424 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:21:10.688517 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:10.688472 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" event={"ID":"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e","Type":"ContainerStarted","Data":"371049b89c6c97b8714fa8c23b34af4557e85e2db8aa1f40deb99bee7b591dee"} Jul 28 17:21:10.688638 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:10.688535 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:21:10.688638 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:10.688564 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:21:10.689997 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:10.689969 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 28 17:21:10.708920 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:10.708850 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" podStartSLOduration=2.898722272 podStartE2EDuration="34.708817008s" totalImagesPullingTime="27.881977885s" totalInitContainerRuntime="3.928116851s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:20:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:21:10.707626842 +0000 UTC m=+653.308079217" watchObservedRunningTime="2026-07-28 17:21:10.708817008 +0000 UTC m=+653.309269336" Jul 28 17:21:10.731918 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:10.731851 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" podStartSLOduration=3.795424847 podStartE2EDuration="34.731836654s" totalImagesPullingTime="27.169436995s" totalInitContainerRuntime="3.766974812s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:20:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:21:10.730121255 +0000 UTC m=+653.330573598" watchObservedRunningTime="2026-07-28 17:21:10.731836654 +0000 UTC m=+653.332289000" Jul 28 17:21:11.693273 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:11.693232 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 28 17:21:11.693730 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:11.693399 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:21:11.694251 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:11.694226 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 28 17:21:12.696624 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:12.696589 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 28 17:21:16.698949 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:16.698913 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:21:16.699483 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:16.699459 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 28 17:21:17.701649 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:17.701621 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:21:17.702261 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:17.702229 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 28 17:21:26.699839 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:26.699793 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 28 17:21:27.702777 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:27.702736 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 28 17:21:36.699634 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:36.699594 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 28 17:21:37.702189 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:37.702145 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 28 17:21:46.699654 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:46.699613 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 28 17:21:47.702515 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:47.702467 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 28 17:21:56.699625 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:56.699580 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 28 17:21:57.702352 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:21:57.702314 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 28 17:22:06.700030 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:22:06.699984 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Jul 28 17:22:07.702697 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:22:07.702655 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 28 17:22:16.700098 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:22:16.700015 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:22:17.702597 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:22:17.702555 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 28 17:22:20.941173 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:22:20.941129 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 28 17:22:30.941685 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:22:30.941654 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:23:06.726695 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:06.726659 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj"] Jul 28 17:23:06.727259 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:06.726930 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kserve-container" containerID="cri-o://69ce2d993a5f0a5ba1e780add82a2c13804ad129f1004b4af24243715f6a1278" gracePeriod=30 Jul 28 17:23:06.727259 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:06.727164 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kube-rbac-proxy" containerID="cri-o://2fbd4e7ca5c360adc6d81e5df48b585bb4c2ac18129531a45c66a25d979566b5" gracePeriod=30 Jul 28 17:23:06.779701 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:06.779668 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z"] Jul 28 17:23:06.779991 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:06.779952 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kserve-container" containerID="cri-o://6c034c63470c2fee2b7d536126eab23e67fd821857d1e84dfe7e1d977f27e201" gracePeriod=30 Jul 28 17:23:06.780127 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:06.780002 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kube-rbac-proxy" containerID="cri-o://371049b89c6c97b8714fa8c23b34af4557e85e2db8aa1f40deb99bee7b591dee" gracePeriod=30 Jul 28 17:23:07.167068 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:07.166983 2562 generic.go:356] "Generic (PLEG): container finished" podID="c369e90f-8cba-427f-8dac-d0669e473abe" containerID="2fbd4e7ca5c360adc6d81e5df48b585bb4c2ac18129531a45c66a25d979566b5" exitCode=2 Jul 28 17:23:07.167068 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:07.167056 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" event={"ID":"c369e90f-8cba-427f-8dac-d0669e473abe","Type":"ContainerDied","Data":"2fbd4e7ca5c360adc6d81e5df48b585bb4c2ac18129531a45c66a25d979566b5"} Jul 28 17:23:07.169110 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:07.169079 2562 generic.go:356] "Generic (PLEG): container finished" podID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerID="371049b89c6c97b8714fa8c23b34af4557e85e2db8aa1f40deb99bee7b591dee" exitCode=2 Jul 28 17:23:07.169228 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:07.169156 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" event={"ID":"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e","Type":"ContainerDied","Data":"371049b89c6c97b8714fa8c23b34af4557e85e2db8aa1f40deb99bee7b591dee"} Jul 28 17:23:07.697318 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:07.697270 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused" Jul 28 17:23:10.617440 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.617412 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:23:10.789750 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.789718 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kserve-provision-location\") pod \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " Jul 28 17:23:10.789750 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.789751 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kube-rbac-proxy-sar-config\") pod \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " Jul 28 17:23:10.789994 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.789804 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-proxy-tls\") pod \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " Jul 28 17:23:10.789994 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.789846 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbdrk\" (UniqueName: \"kubernetes.io/projected/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kube-api-access-jbdrk\") pod \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " Jul 28 17:23:10.789994 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.789862 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-cabundle-cert\") pod \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\" (UID: \"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e\") " Jul 28 17:23:10.790214 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.790121 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" (UID: "ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:23:10.790284 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.790256 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" (UID: "ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:23:10.790369 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.790345 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" (UID: "ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:23:10.792078 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.792047 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kube-api-access-jbdrk" (OuterVolumeSpecName: "kube-api-access-jbdrk") pod "ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" (UID: "ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e"). InnerVolumeSpecName "kube-api-access-jbdrk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:23:10.792156 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.792132 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" (UID: "ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:23:10.891351 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.891319 2562 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kserve-provision-location\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:23:10.891351 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.891350 2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:23:10.891546 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.891362 2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-proxy-tls\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:23:10.891546 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.891375 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbdrk\" (UniqueName: \"kubernetes.io/projected/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-kube-api-access-jbdrk\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:23:10.891546 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.891386 2562 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e-cabundle-cert\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:23:10.941983 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:10.941947 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 28 17:23:11.187736 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.187650 2562 generic.go:356] "Generic (PLEG): container finished" podID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerID="6c034c63470c2fee2b7d536126eab23e67fd821857d1e84dfe7e1d977f27e201" exitCode=0 Jul 28 17:23:11.187736 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.187713 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" Jul 28 17:23:11.187945 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.187730 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" event={"ID":"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e","Type":"ContainerDied","Data":"6c034c63470c2fee2b7d536126eab23e67fd821857d1e84dfe7e1d977f27e201"} Jul 28 17:23:11.187945 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.187797 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z" event={"ID":"ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e","Type":"ContainerDied","Data":"a4480dab4bf70d75f08d8e61edd1552aa5711560e8083e2774bb00969bc91ace"} Jul 28 17:23:11.187945 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.187828 2562 scope.go:117] "RemoveContainer" containerID="371049b89c6c97b8714fa8c23b34af4557e85e2db8aa1f40deb99bee7b591dee" Jul 28 17:23:11.196536 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.196518 2562 scope.go:117] "RemoveContainer" containerID="6c034c63470c2fee2b7d536126eab23e67fd821857d1e84dfe7e1d977f27e201" Jul 28 17:23:11.204135 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.204113 2562 scope.go:117] "RemoveContainer" containerID="e0bbb471572300618f37e8534c89c467fdaf3f99a3dffde553c966e060aa5ade" Jul 28 17:23:11.212089 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.212023 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z"] Jul 28 17:23:11.212163 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.212108 2562 scope.go:117] "RemoveContainer" containerID="371049b89c6c97b8714fa8c23b34af4557e85e2db8aa1f40deb99bee7b591dee" Jul 28 17:23:11.212429 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:23:11.212408 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"371049b89c6c97b8714fa8c23b34af4557e85e2db8aa1f40deb99bee7b591dee\": container with ID starting with 371049b89c6c97b8714fa8c23b34af4557e85e2db8aa1f40deb99bee7b591dee not found: ID does not exist" containerID="371049b89c6c97b8714fa8c23b34af4557e85e2db8aa1f40deb99bee7b591dee" Jul 28 17:23:11.212465 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.212439 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"371049b89c6c97b8714fa8c23b34af4557e85e2db8aa1f40deb99bee7b591dee"} err="failed to get container status \"371049b89c6c97b8714fa8c23b34af4557e85e2db8aa1f40deb99bee7b591dee\": rpc error: code = NotFound desc = could not find container \"371049b89c6c97b8714fa8c23b34af4557e85e2db8aa1f40deb99bee7b591dee\": container with ID starting with 371049b89c6c97b8714fa8c23b34af4557e85e2db8aa1f40deb99bee7b591dee not found: ID does not exist" Jul 28 17:23:11.212465 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.212460 2562 scope.go:117] "RemoveContainer" containerID="6c034c63470c2fee2b7d536126eab23e67fd821857d1e84dfe7e1d977f27e201" Jul 28 17:23:11.212706 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:23:11.212692 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c034c63470c2fee2b7d536126eab23e67fd821857d1e84dfe7e1d977f27e201\": container with ID starting with 6c034c63470c2fee2b7d536126eab23e67fd821857d1e84dfe7e1d977f27e201 not found: ID does not exist" containerID="6c034c63470c2fee2b7d536126eab23e67fd821857d1e84dfe7e1d977f27e201" Jul 28 17:23:11.212751 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.212710 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c034c63470c2fee2b7d536126eab23e67fd821857d1e84dfe7e1d977f27e201"} err="failed to get container status \"6c034c63470c2fee2b7d536126eab23e67fd821857d1e84dfe7e1d977f27e201\": rpc error: code = NotFound desc = could not find container \"6c034c63470c2fee2b7d536126eab23e67fd821857d1e84dfe7e1d977f27e201\": container with ID starting with 6c034c63470c2fee2b7d536126eab23e67fd821857d1e84dfe7e1d977f27e201 not found: ID does not exist" Jul 28 17:23:11.212751 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.212723 2562 scope.go:117] "RemoveContainer" containerID="e0bbb471572300618f37e8534c89c467fdaf3f99a3dffde553c966e060aa5ade" Jul 28 17:23:11.212962 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:23:11.212938 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0bbb471572300618f37e8534c89c467fdaf3f99a3dffde553c966e060aa5ade\": container with ID starting with e0bbb471572300618f37e8534c89c467fdaf3f99a3dffde553c966e060aa5ade not found: ID does not exist" containerID="e0bbb471572300618f37e8534c89c467fdaf3f99a3dffde553c966e060aa5ade" Jul 28 17:23:11.213055 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.212970 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0bbb471572300618f37e8534c89c467fdaf3f99a3dffde553c966e060aa5ade"} err="failed to get container status \"e0bbb471572300618f37e8534c89c467fdaf3f99a3dffde553c966e060aa5ade\": rpc error: code = NotFound desc = could not find container \"e0bbb471572300618f37e8534c89c467fdaf3f99a3dffde553c966e060aa5ade\": container with ID starting with e0bbb471572300618f37e8534c89c467fdaf3f99a3dffde553c966e060aa5ade not found: ID does not exist" Jul 28 17:23:11.216856 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.216835 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-ghf5z"] Jul 28 17:23:11.768112 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.768086 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:23:11.900393 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.900305 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c369e90f-8cba-427f-8dac-d0669e473abe-kube-rbac-proxy-sar-config\") pod \"c369e90f-8cba-427f-8dac-d0669e473abe\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " Jul 28 17:23:11.900393 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.900374 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c369e90f-8cba-427f-8dac-d0669e473abe-kserve-provision-location\") pod \"c369e90f-8cba-427f-8dac-d0669e473abe\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " Jul 28 17:23:11.900601 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.900410 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t6zf\" (UniqueName: \"kubernetes.io/projected/c369e90f-8cba-427f-8dac-d0669e473abe-kube-api-access-9t6zf\") pod \"c369e90f-8cba-427f-8dac-d0669e473abe\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " Jul 28 17:23:11.900601 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.900436 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c369e90f-8cba-427f-8dac-d0669e473abe-proxy-tls\") pod \"c369e90f-8cba-427f-8dac-d0669e473abe\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " Jul 28 17:23:11.900601 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.900478 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c369e90f-8cba-427f-8dac-d0669e473abe-cabundle-cert\") pod \"c369e90f-8cba-427f-8dac-d0669e473abe\" (UID: \"c369e90f-8cba-427f-8dac-d0669e473abe\") " Jul 28 17:23:11.900767 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.900727 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c369e90f-8cba-427f-8dac-d0669e473abe-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "c369e90f-8cba-427f-8dac-d0669e473abe" (UID: "c369e90f-8cba-427f-8dac-d0669e473abe"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:23:11.900864 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.900742 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c369e90f-8cba-427f-8dac-d0669e473abe-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c369e90f-8cba-427f-8dac-d0669e473abe" (UID: "c369e90f-8cba-427f-8dac-d0669e473abe"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:23:11.900973 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.900905 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c369e90f-8cba-427f-8dac-d0669e473abe-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "c369e90f-8cba-427f-8dac-d0669e473abe" (UID: "c369e90f-8cba-427f-8dac-d0669e473abe"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:23:11.902513 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.902494 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c369e90f-8cba-427f-8dac-d0669e473abe-kube-api-access-9t6zf" (OuterVolumeSpecName: "kube-api-access-9t6zf") pod "c369e90f-8cba-427f-8dac-d0669e473abe" (UID: "c369e90f-8cba-427f-8dac-d0669e473abe"). InnerVolumeSpecName "kube-api-access-9t6zf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:23:11.902600 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.902584 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c369e90f-8cba-427f-8dac-d0669e473abe-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c369e90f-8cba-427f-8dac-d0669e473abe" (UID: "c369e90f-8cba-427f-8dac-d0669e473abe"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:23:11.945933 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:11.945904 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" path="/var/lib/kubelet/pods/ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e/volumes" Jul 28 17:23:12.001352 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.001315 2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c369e90f-8cba-427f-8dac-d0669e473abe-proxy-tls\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:23:12.001352 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.001346 2562 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c369e90f-8cba-427f-8dac-d0669e473abe-cabundle-cert\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:23:12.001352 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.001359 2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c369e90f-8cba-427f-8dac-d0669e473abe-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:23:12.001587 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.001373 2562 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c369e90f-8cba-427f-8dac-d0669e473abe-kserve-provision-location\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:23:12.001587 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.001387 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t6zf\" (UniqueName: \"kubernetes.io/projected/c369e90f-8cba-427f-8dac-d0669e473abe-kube-api-access-9t6zf\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 17:23:12.194055 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.193975 2562 generic.go:356] "Generic (PLEG): container finished" podID="c369e90f-8cba-427f-8dac-d0669e473abe" containerID="69ce2d993a5f0a5ba1e780add82a2c13804ad129f1004b4af24243715f6a1278" exitCode=0 Jul 28 17:23:12.194055 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.194039 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" Jul 28 17:23:12.194275 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.194061 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" event={"ID":"c369e90f-8cba-427f-8dac-d0669e473abe","Type":"ContainerDied","Data":"69ce2d993a5f0a5ba1e780add82a2c13804ad129f1004b4af24243715f6a1278"} Jul 28 17:23:12.194275 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.194137 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj" event={"ID":"c369e90f-8cba-427f-8dac-d0669e473abe","Type":"ContainerDied","Data":"b826609b44d8788d5eefe9b00c62842ff1d5cd120c6dee3277c2cfb63cd50652"} Jul 28 17:23:12.194275 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.194181 2562 scope.go:117] "RemoveContainer" containerID="2fbd4e7ca5c360adc6d81e5df48b585bb4c2ac18129531a45c66a25d979566b5" Jul 28 17:23:12.202048 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.202029 2562 scope.go:117] "RemoveContainer" containerID="69ce2d993a5f0a5ba1e780add82a2c13804ad129f1004b4af24243715f6a1278" Jul 28 17:23:12.209142 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.209124 2562 scope.go:117] "RemoveContainer" containerID="c27c752eca262caee3edb387ce7aa7cc548c38908bf651801ed7d04e203428a0" Jul 28 17:23:12.214326 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.214303 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj"] Jul 28 17:23:12.216980 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.216962 2562 scope.go:117] "RemoveContainer" containerID="2fbd4e7ca5c360adc6d81e5df48b585bb4c2ac18129531a45c66a25d979566b5" Jul 28 17:23:12.217242 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:23:12.217222 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fbd4e7ca5c360adc6d81e5df48b585bb4c2ac18129531a45c66a25d979566b5\": container with ID starting with 2fbd4e7ca5c360adc6d81e5df48b585bb4c2ac18129531a45c66a25d979566b5 not found: ID does not exist" containerID="2fbd4e7ca5c360adc6d81e5df48b585bb4c2ac18129531a45c66a25d979566b5" Jul 28 17:23:12.217323 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.217252 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fbd4e7ca5c360adc6d81e5df48b585bb4c2ac18129531a45c66a25d979566b5"} err="failed to get container status \"2fbd4e7ca5c360adc6d81e5df48b585bb4c2ac18129531a45c66a25d979566b5\": rpc error: code = NotFound desc = could not find container \"2fbd4e7ca5c360adc6d81e5df48b585bb4c2ac18129531a45c66a25d979566b5\": container with ID starting with 2fbd4e7ca5c360adc6d81e5df48b585bb4c2ac18129531a45c66a25d979566b5 not found: ID does not exist" Jul 28 17:23:12.217323 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.217278 2562 scope.go:117] "RemoveContainer" containerID="69ce2d993a5f0a5ba1e780add82a2c13804ad129f1004b4af24243715f6a1278" Jul 28 17:23:12.217509 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:23:12.217490 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69ce2d993a5f0a5ba1e780add82a2c13804ad129f1004b4af24243715f6a1278\": container with ID starting with 69ce2d993a5f0a5ba1e780add82a2c13804ad129f1004b4af24243715f6a1278 not found: ID does not exist" containerID="69ce2d993a5f0a5ba1e780add82a2c13804ad129f1004b4af24243715f6a1278" Jul 28 17:23:12.217547 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.217515 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69ce2d993a5f0a5ba1e780add82a2c13804ad129f1004b4af24243715f6a1278"} err="failed to get container status \"69ce2d993a5f0a5ba1e780add82a2c13804ad129f1004b4af24243715f6a1278\": rpc error: code = NotFound desc = could not find container \"69ce2d993a5f0a5ba1e780add82a2c13804ad129f1004b4af24243715f6a1278\": container with ID starting with 69ce2d993a5f0a5ba1e780add82a2c13804ad129f1004b4af24243715f6a1278 not found: ID does not exist" Jul 28 17:23:12.217547 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.217533 2562 scope.go:117] "RemoveContainer" containerID="c27c752eca262caee3edb387ce7aa7cc548c38908bf651801ed7d04e203428a0" Jul 28 17:23:12.217759 ip-10-0-139-193 kubenswrapper[2562]: E0728 17:23:12.217728 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c27c752eca262caee3edb387ce7aa7cc548c38908bf651801ed7d04e203428a0\": container with ID starting with c27c752eca262caee3edb387ce7aa7cc548c38908bf651801ed7d04e203428a0 not found: ID does not exist" containerID="c27c752eca262caee3edb387ce7aa7cc548c38908bf651801ed7d04e203428a0" Jul 28 17:23:12.217853 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.217764 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c27c752eca262caee3edb387ce7aa7cc548c38908bf651801ed7d04e203428a0"} err="failed to get container status \"c27c752eca262caee3edb387ce7aa7cc548c38908bf651801ed7d04e203428a0\": rpc error: code = NotFound desc = could not find container \"c27c752eca262caee3edb387ce7aa7cc548c38908bf651801ed7d04e203428a0\": container with ID starting with c27c752eca262caee3edb387ce7aa7cc548c38908bf651801ed7d04e203428a0 not found: ID does not exist" Jul 28 17:23:12.220516 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:12.220497 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-pmmlj"] Jul 28 17:23:13.945670 ip-10-0-139-193 kubenswrapper[2562]: I0728 17:23:13.945629 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" path="/var/lib/kubelet/pods/c369e90f-8cba-427f-8dac-d0669e473abe/volumes" Jul 28 18:00:31.594385 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.594345 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j62sp/must-gather-792sc"] Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.594781 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kserve-container" Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.594796 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kserve-container" Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.594811 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="storage-initializer" Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.594821 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="storage-initializer" Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.594831 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kube-rbac-proxy" Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.594839 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kube-rbac-proxy" Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.594863 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kserve-container" Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.594885 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kserve-container" Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.594896 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kube-rbac-proxy" Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.594904 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kube-rbac-proxy" Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.594915 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="storage-initializer" Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.594923 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="storage-initializer" Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.594999 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kserve-container" Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.595010 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kserve-container" Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.595020 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c369e90f-8cba-427f-8dac-d0669e473abe" containerName="kube-rbac-proxy" Jul 28 18:00:31.596830 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.595032 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebe8c668-bf03-4eb4-9e6d-5b89f3af8f8e" containerName="kube-rbac-proxy" Jul 28 18:00:31.598131 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.598114 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j62sp/must-gather-792sc" Jul 28 18:00:31.602721 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.602700 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-j62sp\"/\"default-dockercfg-4fm9p\"" Jul 28 18:00:31.603923 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.603902 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j62sp\"/\"kube-root-ca.crt\"" Jul 28 18:00:31.604034 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.603910 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j62sp\"/\"openshift-service-ca.crt\"" Jul 28 18:00:31.609726 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.609706 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j62sp/must-gather-792sc"] Jul 28 18:00:31.613506 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.613481 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9267j\" (UniqueName: \"kubernetes.io/projected/457dc737-b0e6-40af-afe8-56970b7f3196-kube-api-access-9267j\") pod \"must-gather-792sc\" (UID: \"457dc737-b0e6-40af-afe8-56970b7f3196\") " pod="openshift-must-gather-j62sp/must-gather-792sc" Jul 28 18:00:31.613599 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.613521 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/457dc737-b0e6-40af-afe8-56970b7f3196-must-gather-output\") pod \"must-gather-792sc\" (UID: \"457dc737-b0e6-40af-afe8-56970b7f3196\") " pod="openshift-must-gather-j62sp/must-gather-792sc" Jul 28 18:00:31.714769 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.714739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9267j\" (UniqueName: \"kubernetes.io/projected/457dc737-b0e6-40af-afe8-56970b7f3196-kube-api-access-9267j\") pod \"must-gather-792sc\" (UID: \"457dc737-b0e6-40af-afe8-56970b7f3196\") " pod="openshift-must-gather-j62sp/must-gather-792sc" Jul 28 18:00:31.715005 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.714777 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/457dc737-b0e6-40af-afe8-56970b7f3196-must-gather-output\") pod \"must-gather-792sc\" (UID: \"457dc737-b0e6-40af-afe8-56970b7f3196\") " pod="openshift-must-gather-j62sp/must-gather-792sc" Jul 28 18:00:31.715103 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.715087 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/457dc737-b0e6-40af-afe8-56970b7f3196-must-gather-output\") pod \"must-gather-792sc\" (UID: \"457dc737-b0e6-40af-afe8-56970b7f3196\") " pod="openshift-must-gather-j62sp/must-gather-792sc" Jul 28 18:00:31.724181 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.724152 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9267j\" (UniqueName: \"kubernetes.io/projected/457dc737-b0e6-40af-afe8-56970b7f3196-kube-api-access-9267j\") pod \"must-gather-792sc\" (UID: \"457dc737-b0e6-40af-afe8-56970b7f3196\") " pod="openshift-must-gather-j62sp/must-gather-792sc" Jul 28 18:00:31.920563 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:31.920479 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j62sp/must-gather-792sc" Jul 28 18:00:32.046678 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:32.046655 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j62sp/must-gather-792sc"] Jul 28 18:00:32.049036 ip-10-0-139-193 kubenswrapper[2562]: W0728 18:00:32.049006 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod457dc737_b0e6_40af_afe8_56970b7f3196.slice/crio-bd20b85e3fad7b94ea323c97553c26d49ec48eace51f6ea26aec37c91b48218a WatchSource:0}: Error finding container bd20b85e3fad7b94ea323c97553c26d49ec48eace51f6ea26aec37c91b48218a: Status 404 returned error can't find the container with id bd20b85e3fad7b94ea323c97553c26d49ec48eace51f6ea26aec37c91b48218a Jul 28 18:00:32.050723 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:32.050709 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 18:00:32.794531 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:32.794494 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j62sp/must-gather-792sc" event={"ID":"457dc737-b0e6-40af-afe8-56970b7f3196","Type":"ContainerStarted","Data":"bd20b85e3fad7b94ea323c97553c26d49ec48eace51f6ea26aec37c91b48218a"} Jul 28 18:00:38.823237 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:38.823187 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j62sp/must-gather-792sc" event={"ID":"457dc737-b0e6-40af-afe8-56970b7f3196","Type":"ContainerStarted","Data":"423a3974364a079a183a9374e3f19ab9e68a3d85feeffcfa529131eba20b4949"} Jul 28 18:00:38.823237 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:38.823231 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j62sp/must-gather-792sc" event={"ID":"457dc737-b0e6-40af-afe8-56970b7f3196","Type":"ContainerStarted","Data":"ac45dfd93fb1b50789f2f7782c9ff3ab6fc31824e0cef18d94ecda0aeadec505"} Jul 28 18:00:38.842688 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:00:38.842611 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-j62sp/must-gather-792sc" podStartSLOduration=1.8751758710000002 podStartE2EDuration="7.842587436s" totalImagesPullingTime="5.967411565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:00:38.840955728 +0000 UTC m=+3021.441408086" watchObservedRunningTime="2026-07-28 18:00:38.842587436 +0000 UTC m=+3021.443039783" Jul 28 18:01:01.928173 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:01.928139 2562 generic.go:356] "Generic (PLEG): container finished" podID="457dc737-b0e6-40af-afe8-56970b7f3196" containerID="ac45dfd93fb1b50789f2f7782c9ff3ab6fc31824e0cef18d94ecda0aeadec505" exitCode=0 Jul 28 18:01:01.928616 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:01.928210 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j62sp/must-gather-792sc" event={"ID":"457dc737-b0e6-40af-afe8-56970b7f3196","Type":"ContainerDied","Data":"ac45dfd93fb1b50789f2f7782c9ff3ab6fc31824e0cef18d94ecda0aeadec505"} Jul 28 18:01:01.928616 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:01.928568 2562 scope.go:117] "RemoveContainer" containerID="ac45dfd93fb1b50789f2f7782c9ff3ab6fc31824e0cef18d94ecda0aeadec505" Jul 28 18:01:02.350177 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:02.350144 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j62sp_must-gather-792sc_457dc737-b0e6-40af-afe8-56970b7f3196/gather/0.log" Jul 28 18:01:05.536396 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:05.536362 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xn76m_4f49cd3a-ee50-43ee-a297-ca95c775c799/global-pull-secret-syncer/0.log" Jul 28 18:01:05.674593 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:05.674559 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-s4ggt_4a894b9f-5c78-4ca0-9375-8b6465b5d5bc/konnectivity-agent/0.log" Jul 28 18:01:05.767209 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:05.767173 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-193.ec2.internal_0e753e1f8e3d27215843b4f99d76207a/haproxy/0.log" Jul 28 18:01:07.832619 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:07.832578 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-j62sp/must-gather-792sc"] Jul 28 18:01:07.833152 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:07.832828 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-j62sp/must-gather-792sc" podUID="457dc737-b0e6-40af-afe8-56970b7f3196" containerName="copy" containerID="cri-o://423a3974364a079a183a9374e3f19ab9e68a3d85feeffcfa529131eba20b4949" gracePeriod=2 Jul 28 18:01:07.835893 ip-10-0-139-193 kubenswrapper[2562]: E0728 18:01:07.835829 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-792sc\" is forbidden: User \"system:node:ip-10-0-139-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-j62sp\": no relationship found between node 'ip-10-0-139-193.ec2.internal' and this object" podUID="457dc737-b0e6-40af-afe8-56970b7f3196" pod="openshift-must-gather-j62sp/must-gather-792sc" Jul 28 18:01:07.837061 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:07.836728 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-j62sp/must-gather-792sc"] Jul 28 18:01:07.946241 ip-10-0-139-193 kubenswrapper[2562]: E0728 18:01:07.946206 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-792sc\" is forbidden: User \"system:node:ip-10-0-139-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-j62sp\": no relationship found between node 'ip-10-0-139-193.ec2.internal' and this object" podUID="457dc737-b0e6-40af-afe8-56970b7f3196" pod="openshift-must-gather-j62sp/must-gather-792sc" Jul 28 18:01:07.954665 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:07.954642 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j62sp_must-gather-792sc_457dc737-b0e6-40af-afe8-56970b7f3196/copy/0.log" Jul 28 18:01:07.955108 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:07.955072 2562 generic.go:356] "Generic (PLEG): container finished" podID="457dc737-b0e6-40af-afe8-56970b7f3196" containerID="423a3974364a079a183a9374e3f19ab9e68a3d85feeffcfa529131eba20b4949" exitCode=143 Jul 28 18:01:08.063731 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:08.063707 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j62sp_must-gather-792sc_457dc737-b0e6-40af-afe8-56970b7f3196/copy/0.log" Jul 28 18:01:08.065466 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:08.065449 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j62sp/must-gather-792sc" Jul 28 18:01:08.242650 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:08.242616 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9267j\" (UniqueName: \"kubernetes.io/projected/457dc737-b0e6-40af-afe8-56970b7f3196-kube-api-access-9267j\") pod \"457dc737-b0e6-40af-afe8-56970b7f3196\" (UID: \"457dc737-b0e6-40af-afe8-56970b7f3196\") " Jul 28 18:01:08.242650 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:08.242653 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/457dc737-b0e6-40af-afe8-56970b7f3196-must-gather-output\") pod \"457dc737-b0e6-40af-afe8-56970b7f3196\" (UID: \"457dc737-b0e6-40af-afe8-56970b7f3196\") " Jul 28 18:01:08.244210 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:08.244179 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/457dc737-b0e6-40af-afe8-56970b7f3196-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "457dc737-b0e6-40af-afe8-56970b7f3196" (UID: "457dc737-b0e6-40af-afe8-56970b7f3196"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 18:01:08.244868 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:08.244842 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/457dc737-b0e6-40af-afe8-56970b7f3196-kube-api-access-9267j" (OuterVolumeSpecName: "kube-api-access-9267j") pod "457dc737-b0e6-40af-afe8-56970b7f3196" (UID: "457dc737-b0e6-40af-afe8-56970b7f3196"). InnerVolumeSpecName "kube-api-access-9267j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 18:01:08.343218 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:08.343183 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9267j\" (UniqueName: \"kubernetes.io/projected/457dc737-b0e6-40af-afe8-56970b7f3196-kube-api-access-9267j\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 18:01:08.343218 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:08.343216 2562 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/457dc737-b0e6-40af-afe8-56970b7f3196-must-gather-output\") on node \"ip-10-0-139-193.ec2.internal\" DevicePath \"\"" Jul 28 18:01:08.960927 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:08.960898 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j62sp_must-gather-792sc_457dc737-b0e6-40af-afe8-56970b7f3196/copy/0.log" Jul 28 18:01:08.961355 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:08.961265 2562 scope.go:117] "RemoveContainer" containerID="423a3974364a079a183a9374e3f19ab9e68a3d85feeffcfa529131eba20b4949" Jul 28 18:01:08.961355 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:08.961304 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j62sp/must-gather-792sc" Jul 28 18:01:08.969492 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:08.969156 2562 scope.go:117] "RemoveContainer" containerID="ac45dfd93fb1b50789f2f7782c9ff3ab6fc31824e0cef18d94ecda0aeadec505" Jul 28 18:01:09.930222 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:09.930102 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-sxlc5_67735dc9-f5eb-4144-9814-9e899af24e05/kube-state-metrics/0.log" Jul 28 18:01:09.946889 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:09.946835 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="457dc737-b0e6-40af-afe8-56970b7f3196" path="/var/lib/kubelet/pods/457dc737-b0e6-40af-afe8-56970b7f3196/volumes" Jul 28 18:01:09.968325 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:09.968297 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-sxlc5_67735dc9-f5eb-4144-9814-9e899af24e05/kube-rbac-proxy-main/0.log" Jul 28 18:01:10.003583 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.003545 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-sxlc5_67735dc9-f5eb-4144-9814-9e899af24e05/kube-rbac-proxy-self/0.log" Jul 28 18:01:10.041598 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.041563 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5c779bb47b-8tx5c_7765c642-f896-4490-bddc-5c10f97fc55a/metrics-server/0.log" Jul 28 18:01:10.234916 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.234870 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wspw8_d3159c85-7f2a-4e8a-b074-e177e9dbfd12/node-exporter/0.log" Jul 28 18:01:10.261704 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.261658 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wspw8_d3159c85-7f2a-4e8a-b074-e177e9dbfd12/kube-rbac-proxy/0.log" Jul 28 18:01:10.289847 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.289819 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wspw8_d3159c85-7f2a-4e8a-b074-e177e9dbfd12/init-textfile/0.log" Jul 28 18:01:10.417038 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.417012 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-2wj2h_4a7d976b-5549-4246-be0b-64772c7ca3a8/kube-rbac-proxy-main/0.log" Jul 28 18:01:10.443940 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.443907 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-2wj2h_4a7d976b-5549-4246-be0b-64772c7ca3a8/kube-rbac-proxy-self/0.log" Jul 28 18:01:10.471412 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.471370 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-2wj2h_4a7d976b-5549-4246-be0b-64772c7ca3a8/openshift-state-metrics/0.log" Jul 28 18:01:10.529576 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.529498 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fab3ebaa-7846-41e4-ab5a-5390fbad1cec/prometheus/0.log" Jul 28 18:01:10.553391 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.553353 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fab3ebaa-7846-41e4-ab5a-5390fbad1cec/config-reloader/0.log" Jul 28 18:01:10.584945 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.584913 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fab3ebaa-7846-41e4-ab5a-5390fbad1cec/thanos-sidecar/0.log" Jul 28 18:01:10.620777 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.620750 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fab3ebaa-7846-41e4-ab5a-5390fbad1cec/kube-rbac-proxy-web/0.log" Jul 28 18:01:10.649736 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.649710 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fab3ebaa-7846-41e4-ab5a-5390fbad1cec/kube-rbac-proxy/0.log" Jul 28 18:01:10.678170 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.678132 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fab3ebaa-7846-41e4-ab5a-5390fbad1cec/kube-rbac-proxy-thanos/0.log" Jul 28 18:01:10.708502 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.708472 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fab3ebaa-7846-41e4-ab5a-5390fbad1cec/init-config-reloader/0.log" Jul 28 18:01:10.745899 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.745860 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-65k7t_e160931f-1ab3-488c-8788-ad4b0ca2083e/prometheus-operator/0.log" Jul 28 18:01:10.770775 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.770746 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-65k7t_e160931f-1ab3-488c-8788-ad4b0ca2083e/kube-rbac-proxy/0.log" Jul 28 18:01:10.813325 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:10.813236 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-t4kz8_2e004692-883b-4311-acec-06713b142f67/prometheus-operator-admission-webhook/0.log" Jul 28 18:01:13.057127 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.057089 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-d7xlp_baa763a5-35b5-4ab4-8a81-54c23b7de388/console-operator/0.log" Jul 28 18:01:13.465649 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.465567 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj"] Jul 28 18:01:13.465895 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.465867 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="457dc737-b0e6-40af-afe8-56970b7f3196" containerName="gather" Jul 28 18:01:13.465895 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.465895 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="457dc737-b0e6-40af-afe8-56970b7f3196" containerName="gather" Jul 28 18:01:13.465978 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.465915 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="457dc737-b0e6-40af-afe8-56970b7f3196" containerName="copy" Jul 28 18:01:13.465978 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.465921 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="457dc737-b0e6-40af-afe8-56970b7f3196" containerName="copy" Jul 28 18:01:13.465978 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.465964 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="457dc737-b0e6-40af-afe8-56970b7f3196" containerName="gather" Jul 28 18:01:13.465978 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.465972 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="457dc737-b0e6-40af-afe8-56970b7f3196" containerName="copy" Jul 28 18:01:13.469513 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.469486 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.472816 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.472796 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ck55s\"/\"kube-root-ca.crt\"" Jul 28 18:01:13.472940 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.472919 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ck55s\"/\"default-dockercfg-p6qwz\"" Jul 28 18:01:13.474086 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.474069 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ck55s\"/\"openshift-service-ca.crt\"" Jul 28 18:01:13.478183 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.478158 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj"] Jul 28 18:01:13.549415 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.549382 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-hblt4_86e8f2ce-1d58-4acc-8d6e-fbb4bc8d208c/download-server/0.log" Jul 28 18:01:13.585560 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.585523 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6tbj\" (UniqueName: \"kubernetes.io/projected/fb4b66d6-592e-4439-b06c-c950256aafb2-kube-api-access-n6tbj\") pod \"perf-node-gather-daemonset-4jbrj\" (UID: \"fb4b66d6-592e-4439-b06c-c950256aafb2\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.585736 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.585575 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb4b66d6-592e-4439-b06c-c950256aafb2-sys\") pod \"perf-node-gather-daemonset-4jbrj\" (UID: \"fb4b66d6-592e-4439-b06c-c950256aafb2\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.585736 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.585598 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb4b66d6-592e-4439-b06c-c950256aafb2-proc\") pod \"perf-node-gather-daemonset-4jbrj\" (UID: \"fb4b66d6-592e-4439-b06c-c950256aafb2\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.585736 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.585633 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb4b66d6-592e-4439-b06c-c950256aafb2-lib-modules\") pod \"perf-node-gather-daemonset-4jbrj\" (UID: \"fb4b66d6-592e-4439-b06c-c950256aafb2\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.585736 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.585661 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb4b66d6-592e-4439-b06c-c950256aafb2-podres\") pod \"perf-node-gather-daemonset-4jbrj\" (UID: \"fb4b66d6-592e-4439-b06c-c950256aafb2\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.686678 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.686638 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb4b66d6-592e-4439-b06c-c950256aafb2-sys\") pod \"perf-node-gather-daemonset-4jbrj\" (UID: \"fb4b66d6-592e-4439-b06c-c950256aafb2\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.686678 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.686675 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb4b66d6-592e-4439-b06c-c950256aafb2-proc\") pod \"perf-node-gather-daemonset-4jbrj\" (UID: \"fb4b66d6-592e-4439-b06c-c950256aafb2\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.686935 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.686694 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb4b66d6-592e-4439-b06c-c950256aafb2-lib-modules\") pod \"perf-node-gather-daemonset-4jbrj\" (UID: \"fb4b66d6-592e-4439-b06c-c950256aafb2\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.686935 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.686739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb4b66d6-592e-4439-b06c-c950256aafb2-podres\") pod \"perf-node-gather-daemonset-4jbrj\" (UID: \"fb4b66d6-592e-4439-b06c-c950256aafb2\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.686935 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.686774 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb4b66d6-592e-4439-b06c-c950256aafb2-proc\") pod \"perf-node-gather-daemonset-4jbrj\" (UID: \"fb4b66d6-592e-4439-b06c-c950256aafb2\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.686935 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.686771 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb4b66d6-592e-4439-b06c-c950256aafb2-sys\") pod \"perf-node-gather-daemonset-4jbrj\" (UID: \"fb4b66d6-592e-4439-b06c-c950256aafb2\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.686935 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.686805 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6tbj\" (UniqueName: \"kubernetes.io/projected/fb4b66d6-592e-4439-b06c-c950256aafb2-kube-api-access-n6tbj\") pod \"perf-node-gather-daemonset-4jbrj\" (UID: \"fb4b66d6-592e-4439-b06c-c950256aafb2\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.686935 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.686923 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb4b66d6-592e-4439-b06c-c950256aafb2-podres\") pod \"perf-node-gather-daemonset-4jbrj\" (UID: \"fb4b66d6-592e-4439-b06c-c950256aafb2\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.687123 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.686934 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb4b66d6-592e-4439-b06c-c950256aafb2-lib-modules\") pod \"perf-node-gather-daemonset-4jbrj\" (UID: \"fb4b66d6-592e-4439-b06c-c950256aafb2\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.695849 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.695820 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6tbj\" (UniqueName: \"kubernetes.io/projected/fb4b66d6-592e-4439-b06c-c950256aafb2-kube-api-access-n6tbj\") pod \"perf-node-gather-daemonset-4jbrj\" (UID: \"fb4b66d6-592e-4439-b06c-c950256aafb2\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.779949 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.779915 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:13.910458 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.910432 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj"] Jul 28 18:01:13.913459 ip-10-0-139-193 kubenswrapper[2562]: W0728 18:01:13.913433 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfb4b66d6_592e_4439_b06c_c950256aafb2.slice/crio-2cb53064a4c16d144fccc8f60c0d353e5704d62f7aeadc202cb0387d9db2bca9 WatchSource:0}: Error finding container 2cb53064a4c16d144fccc8f60c0d353e5704d62f7aeadc202cb0387d9db2bca9: Status 404 returned error can't find the container with id 2cb53064a4c16d144fccc8f60c0d353e5704d62f7aeadc202cb0387d9db2bca9 Jul 28 18:01:13.982620 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.982589 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" event={"ID":"fb4b66d6-592e-4439-b06c-c950256aafb2","Type":"ContainerStarted","Data":"f90dbf61a0ef006dabf90d023f4ea9ee849108bad49d6999862e6309143e2a79"} Jul 28 18:01:13.982777 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.982642 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" event={"ID":"fb4b66d6-592e-4439-b06c-c950256aafb2","Type":"ContainerStarted","Data":"2cb53064a4c16d144fccc8f60c0d353e5704d62f7aeadc202cb0387d9db2bca9"} Jul 28 18:01:13.982777 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:13.982668 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:14.001449 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:14.001399 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" podStartSLOduration=1.001384215 podStartE2EDuration="1.001384215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:01:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:14.000801896 +0000 UTC m=+3056.601254267" watchObservedRunningTime="2026-07-28 18:01:14.001384215 +0000 UTC m=+3056.601836561" Jul 28 18:01:14.104105 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:14.104015 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-m96t9_c4dae54a-3fc7-4b5b-aa8e-c08155e12eca/volume-data-source-validator/0.log" Jul 28 18:01:15.079082 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:15.079056 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9m9d6_34c245a9-ee7b-4561-aebf-43c4c325c332/dns/0.log" Jul 28 18:01:15.109255 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:15.109218 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9m9d6_34c245a9-ee7b-4561-aebf-43c4c325c332/kube-rbac-proxy/0.log" Jul 28 18:01:15.248008 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:15.247975 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-frb7f_249db02c-8b5f-42ba-922e-871de558c0b1/dns-node-resolver/0.log" Jul 28 18:01:15.836346 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:15.836317 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w8nl6_ef9a5eff-04c8-4af5-b00f-3dc2f5d0955d/node-ca/0.log" Jul 28 18:01:16.590676 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:16.590646 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d944fd6b9-xmtww_5d779727-dc91-4c3d-863a-622cbfdfbd47/router/0.log" Jul 28 18:01:17.009248 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:17.009214 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qqpxl_c0218446-ab4e-4002-9f70-1d5c5c222bed/serve-healthcheck-canary/0.log" Jul 28 18:01:17.414318 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:17.414233 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-q5w5v_a72944f0-79e5-407d-9add-0c510b4bc4a8/insights-operator/0.log" Jul 28 18:01:17.520733 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:17.520704 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s7zwq_ec7b5a58-1942-455b-8195-c72e4c57e316/kube-rbac-proxy/0.log" Jul 28 18:01:17.547097 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:17.547065 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s7zwq_ec7b5a58-1942-455b-8195-c72e4c57e316/exporter/0.log" Jul 28 18:01:17.574407 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:17.574379 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s7zwq_ec7b5a58-1942-455b-8195-c72e4c57e316/extractor/0.log" Jul 28 18:01:19.995604 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:19.995571 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-4jbrj" Jul 28 18:01:20.098010 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:20.097977 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-bwcxm_54358545-3801-44f1-952e-5aba029ea28b/s3-init/0.log" Jul 28 18:01:20.126813 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:20.126780 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-bwcxm_54358545-3801-44f1-952e-5aba029ea28b/download-hf-model/0.log" Jul 28 18:01:26.468215 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:26.468184 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zs8m6_3f00b9d9-ad3e-4e58-8faa-da2eed76d38e/kube-multus-additional-cni-plugins/0.log" Jul 28 18:01:26.497464 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:26.497432 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zs8m6_3f00b9d9-ad3e-4e58-8faa-da2eed76d38e/egress-router-binary-copy/0.log" Jul 28 18:01:26.526404 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:26.526374 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zs8m6_3f00b9d9-ad3e-4e58-8faa-da2eed76d38e/cni-plugins/0.log" Jul 28 18:01:26.574541 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:26.574511 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zs8m6_3f00b9d9-ad3e-4e58-8faa-da2eed76d38e/bond-cni-plugin/0.log" Jul 28 18:01:26.625764 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:26.625732 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zs8m6_3f00b9d9-ad3e-4e58-8faa-da2eed76d38e/routeoverride-cni/0.log" Jul 28 18:01:26.673064 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:26.673031 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zs8m6_3f00b9d9-ad3e-4e58-8faa-da2eed76d38e/whereabouts-cni-bincopy/0.log" Jul 28 18:01:26.716770 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:26.716741 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zs8m6_3f00b9d9-ad3e-4e58-8faa-da2eed76d38e/whereabouts-cni/0.log" Jul 28 18:01:26.825027 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:26.824997 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xfvwc_033fbb73-fda9-44a7-975b-edf75e79f4b1/kube-multus/0.log" Jul 28 18:01:27.062706 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:27.062674 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nq44j_64efaaa1-f45f-431e-a2d7-9b05a714971e/network-metrics-daemon/0.log" Jul 28 18:01:27.106214 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:27.106129 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nq44j_64efaaa1-f45f-431e-a2d7-9b05a714971e/kube-rbac-proxy/0.log" Jul 28 18:01:28.688890 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:28.688853 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxpf9_9bc3a85a-7d3c-45dd-a991-1a0f8549dc57/ovn-controller/0.log" Jul 28 18:01:28.729329 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:28.729294 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxpf9_9bc3a85a-7d3c-45dd-a991-1a0f8549dc57/ovn-acl-logging/0.log" Jul 28 18:01:28.772165 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:28.772128 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxpf9_9bc3a85a-7d3c-45dd-a991-1a0f8549dc57/kube-rbac-proxy-node/0.log" Jul 28 18:01:28.811495 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:28.811466 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxpf9_9bc3a85a-7d3c-45dd-a991-1a0f8549dc57/kube-rbac-proxy-ovn-metrics/0.log" Jul 28 18:01:28.853921 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:28.853889 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxpf9_9bc3a85a-7d3c-45dd-a991-1a0f8549dc57/northd/0.log" Jul 28 18:01:28.897485 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:28.897455 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxpf9_9bc3a85a-7d3c-45dd-a991-1a0f8549dc57/nbdb/0.log" Jul 28 18:01:28.946509 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:28.946428 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxpf9_9bc3a85a-7d3c-45dd-a991-1a0f8549dc57/sbdb/0.log" Jul 28 18:01:29.030104 ip-10-0-139-193 kubenswrapper[2562]: I0728 18:01:29.030072 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kxpf9_9bc3a85a-7d3c-45dd-a991-1a0f8549dc57/ovnkube-controller/0.log"