Jul 09 18:41:23.128918 ip-10-0-143-175 systemd[1]: Starting Kubernetes Kubelet... Jul 09 18:41:23.584111 ip-10-0-143-175 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 18:41:23.584111 ip-10-0-143-175 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 18:41:23.584111 ip-10-0-143-175 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 18:41:23.584111 ip-10-0-143-175 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 18:41:23.584111 ip-10-0-143-175 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 18:41:23.584946 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.584175 2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 18:41:23.590476 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590312 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:41:23.590476 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590471 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:41:23.590476 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590477 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:41:23.590476 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590481 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:41:23.590476 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590484 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:41:23.590476 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590488 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590491 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590494 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590497 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590500 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590503 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590505 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590509 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590512 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590514 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590517 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590521 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590524 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590526 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590529 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590534 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590540 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590543 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590546 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:41:23.590756 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590549 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590552 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590555 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590557 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590560 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590562 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590565 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590567 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590570 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590573 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590575 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590579 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590599 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590603 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590608 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590612 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590614 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590617 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590620 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590622 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:41:23.591211 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590625 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590628 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590630 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590633 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590635 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590638 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590641 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590644 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590647 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590652 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590654 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590657 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590660 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590662 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590665 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590668 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590671 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590673 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590676 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590679 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:41:23.591725 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590681 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590684 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590686 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590689 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590692 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590694 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590698 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590700 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590706 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590711 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590715 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590718 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590721 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590724 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590727 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590729 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590732 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590735 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590738 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590742 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:41:23.592208 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590745 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590748 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590750 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590753 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590755 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590758 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590760 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590763 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590766 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590768 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590771 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590774 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590776 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590779 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.590781 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591511 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591519 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591522 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591526 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:41:23.592744 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591530 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591533 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591536 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591540 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591543 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591546 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591549 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591552 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591555 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591557 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591560 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591562 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591565 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591568 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591571 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591573 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591576 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591578 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591600 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591603 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:41:23.593213 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591606 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591608 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591611 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591614 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591616 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591621 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591624 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591627 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591629 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591632 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591635 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591638 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591641 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591643 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591646 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591649 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591651 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591654 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591656 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:41:23.593734 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591659 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591661 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591663 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591666 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591668 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591671 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591673 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591678 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591681 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591685 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591687 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591690 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591692 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591694 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591697 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591699 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591702 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591704 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591707 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591710 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:41:23.594291 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591712 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591715 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591717 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591719 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591722 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591725 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591728 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591731 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591734 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591737 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591740 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591742 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591745 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591747 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591750 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591753 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591756 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591758 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591761 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:41:23.594813 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591763 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591767 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591769 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591771 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591774 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591776 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591779 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591781 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591784 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591786 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591789 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591791 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591794 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591797 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591799 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591801 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.591804 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592190 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592195 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592199 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592201 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:41:23.595283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592204 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592209 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592212 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592216 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592219 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592222 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592225 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592228 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592231 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592233 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592236 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592240 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592243 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592246 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592248 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592251 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592253 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592256 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592258 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:41:23.595828 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592261 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592263 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592268 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592270 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592273 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592275 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592278 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592280 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592283 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592286 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592288 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592291 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592293 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592295 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592298 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592300 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592302 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592305 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592307 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592309 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:41:23.596283 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592313 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592315 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592319 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592322 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592324 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592327 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592330 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592333 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592336 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592338 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592341 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592344 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592346 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592349 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592352 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592355 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592357 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592360 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592362 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:41:23.596830 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592364 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592367 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592369 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592372 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592374 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592377 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592379 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592382 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592384 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592386 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592389 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592391 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592394 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592396 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592398 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592401 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592404 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592406 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592408 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592411 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:41:23.597315 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592414 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592417 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592419 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592421 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592424 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592426 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592428 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592432 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592434 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592437 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592439 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592441 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592444 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592446 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592449 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592451 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.592453 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592530 2585 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592538 2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592545 2585 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592549 2585 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 18:41:23.597840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592553 2585 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592557 2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592561 2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592565 2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592568 2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592571 2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592575 2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592578 2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592600 2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592604 2585 flags.go:64] FLAG: --cgroup-root="" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592607 2585 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592610 2585 flags.go:64] FLAG: --client-ca-file="" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592614 2585 flags.go:64] FLAG: --cloud-provider="external" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592617 2585 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592621 2585 flags.go:64] FLAG: --cluster-domain="" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592623 2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592626 2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592629 2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592634 2585 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592638 2585 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592641 2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592644 2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592647 2585 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592650 2585 flags.go:64] FLAG: --contention-profiling="false" Jul 09 18:41:23.598378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592653 2585 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592656 2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592659 2585 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592662 2585 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592666 2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592669 2585 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592672 2585 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592674 2585 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592677 2585 flags.go:64] FLAG: --enable-server="true" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592680 2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592685 2585 flags.go:64] FLAG: --event-burst="100" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592687 2585 flags.go:64] FLAG: --event-qps="50" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592690 2585 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592693 2585 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592696 2585 flags.go:64] FLAG: --eviction-hard="" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592699 2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592702 2585 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592705 2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592708 2585 flags.go:64] FLAG: --eviction-soft="" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592710 2585 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592713 2585 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592717 2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592720 2585 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592722 2585 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592725 2585 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 18:41:23.598968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592728 2585 flags.go:64] FLAG: --feature-gates="" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592734 2585 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592737 2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592740 2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592743 2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592746 2585 flags.go:64] FLAG: --healthz-port="10248" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592750 2585 flags.go:64] FLAG: --help="false" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592753 2585 flags.go:64] FLAG: --hostname-override="ip-10-0-143-175.ec2.internal" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592756 2585 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592759 2585 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592771 2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592775 2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592778 2585 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592781 2585 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592784 2585 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592787 2585 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592789 2585 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592792 2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592795 2585 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592798 2585 flags.go:64] FLAG: --kube-reserved="" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592800 2585 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592803 2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592806 2585 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592809 2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 18:41:23.599621 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592811 2585 flags.go:64] FLAG: --lock-file="" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592814 2585 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592817 2585 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592819 2585 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592825 2585 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592831 2585 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592834 2585 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592837 2585 flags.go:64] FLAG: --logging-format="text" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592840 2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592844 2585 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592847 2585 flags.go:64] FLAG: --manifest-url="" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592849 2585 flags.go:64] FLAG: --manifest-url-header="" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592854 2585 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592857 2585 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592862 2585 flags.go:64] FLAG: --max-pods="110" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592865 2585 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592868 2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592871 2585 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592874 2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592876 2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592879 2585 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592882 2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592889 2585 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592892 2585 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592894 2585 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 18:41:23.600209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592897 2585 flags.go:64] FLAG: --pod-cidr="" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592900 2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592906 2585 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592908 2585 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592911 2585 flags.go:64] FLAG: --pods-per-core="0" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592914 2585 flags.go:64] FLAG: --port="10250" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592917 2585 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592920 2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d21feaff50e0bff0" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592922 2585 flags.go:64] FLAG: --qos-reserved="" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592925 2585 flags.go:64] FLAG: --read-only-port="10255" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592928 2585 flags.go:64] FLAG: --register-node="true" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592931 2585 flags.go:64] FLAG: --register-with-taints="" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592935 2585 flags.go:64] FLAG: --registry-burst="10" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592939 2585 flags.go:64] FLAG: --registry-qps="5" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592941 2585 flags.go:64] FLAG: --reserved-cpus="" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592944 2585 flags.go:64] FLAG: --reserved-memory="" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592949 2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592953 2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592956 2585 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592958 2585 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592961 2585 flags.go:64] FLAG: --runonce="false" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592964 2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592968 2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592971 2585 flags.go:64] FLAG: --seccomp-default="false" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592974 2585 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 18:41:23.600850 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592977 2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592980 2585 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592983 2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592986 2585 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592989 2585 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592992 2585 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592994 2585 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.592997 2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593000 2585 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593003 2585 flags.go:64] FLAG: --system-cgroups="" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593005 2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593011 2585 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593014 2585 flags.go:64] FLAG: --tls-cert-file="" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593017 2585 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593021 2585 flags.go:64] FLAG: --tls-min-version="" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593023 2585 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593026 2585 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593029 2585 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593032 2585 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593035 2585 flags.go:64] FLAG: --v="2" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593038 2585 flags.go:64] FLAG: --version="false" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593043 2585 flags.go:64] FLAG: --vmodule="" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593046 2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593054 2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593190 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:41:23.601441 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593194 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593198 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593201 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593203 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593207 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593210 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593212 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593215 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593218 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593220 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593223 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593226 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593229 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593232 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593234 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593237 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593239 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593242 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593244 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593247 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:41:23.602092 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593250 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593253 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593257 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593260 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593262 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593264 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593267 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593270 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593272 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593275 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593279 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593282 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593284 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593287 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593289 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593292 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593296 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593299 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593301 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:41:23.602594 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593304 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593306 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593309 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593311 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593313 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593316 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593318 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593321 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593323 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593325 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593328 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593330 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593333 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593335 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593338 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593340 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593343 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593345 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593348 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:41:23.603066 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593351 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593355 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593359 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593362 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593366 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593369 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593372 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593374 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593376 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593379 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593382 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593385 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593387 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593390 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593392 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593394 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593397 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593399 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593402 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593404 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:41:23.603514 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593407 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593409 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593412 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593415 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593417 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593420 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593422 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593424 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593427 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593429 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593432 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593434 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593437 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593439 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593441 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593444 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593447 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593450 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593452 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:41:23.604026 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.593454 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:41:23.604494 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.593459 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 18:41:23.604494 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.601792 2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 09 18:41:23.604494 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.601811 2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 18:41:23.604494 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.602686 2585 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 18:41:23.604494 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.602694 2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 18:41:23.604494 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602750 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:41:23.604494 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602755 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:41:23.604494 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602759 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:41:23.604494 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602762 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:41:23.604494 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602765 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:41:23.604494 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602767 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:41:23.604494 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602770 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:41:23.604494 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602773 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:41:23.604494 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602776 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:41:23.604494 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602778 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602781 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602783 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602786 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602788 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602791 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602794 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602796 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602799 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602801 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602803 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602806 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602808 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602811 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602814 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602817 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602820 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602823 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602825 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:41:23.604907 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602828 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602831 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602834 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602836 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602839 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602842 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602844 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602847 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602850 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602852 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602855 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602857 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602860 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602863 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602865 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602868 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602871 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602874 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602876 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602878 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:41:23.605387 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602881 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602885 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602889 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602892 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602895 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602897 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602900 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602903 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602906 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602910 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602912 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602914 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602918 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602921 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602923 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602926 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602928 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602930 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602933 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:41:23.605994 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602937 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602940 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602943 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602945 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602948 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602952 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602955 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602958 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602960 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602963 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602966 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602968 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602970 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602973 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602976 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602978 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602981 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602983 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602985 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602988 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:41:23.606460 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602990 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:41:23.606952 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602993 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:41:23.606952 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602995 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:41:23.606952 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.602999 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:41:23.606952 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603002 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:41:23.606952 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603005 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:41:23.606952 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603007 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:41:23.606952 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603010 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:41:23.606952 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603012 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:41:23.606952 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603015 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:41:23.606952 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603017 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:41:23.606952 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603020 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:41:23.606952 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.603025 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 18:41:23.606952 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603138 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:41:23.606952 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603142 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:41:23.606952 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603145 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603148 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603151 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603153 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603156 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603158 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603161 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603164 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603166 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603169 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603171 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603173 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603176 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603178 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603180 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603183 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603185 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603188 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603190 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603192 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:41:23.607324 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603195 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603198 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603200 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603203 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603205 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603208 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603210 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603213 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603215 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603218 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603220 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603223 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603226 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603230 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603232 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603235 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603237 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603240 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603243 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:41:23.607806 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603245 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603247 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603250 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603252 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603255 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603257 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603259 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603262 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603264 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603268 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603271 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603274 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603277 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603279 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603282 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603286 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603289 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603292 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603295 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603297 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:41:23.608265 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603300 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603303 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603306 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603309 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603311 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603314 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603316 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603318 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603321 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603324 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603326 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603329 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603331 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603333 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603336 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603338 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603340 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603343 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603345 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:41:23.608791 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603349 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603352 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603355 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603357 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603360 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603362 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603365 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603368 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603370 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603373 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603376 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603379 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603381 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603383 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603386 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603388 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603390 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603392 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:41:23.609251 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:23.603395 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:41:23.609696 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.603400 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 18:41:23.609696 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.603511 2585 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 18:41:23.609696 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.605932 2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 18:41:23.609696 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.607649 2585 server.go:1019] "Starting client certificate rotation" Jul 09 18:41:23.609696 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.607751 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 18:41:23.609696 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.607784 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 18:41:23.610578 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.610564 2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 18:41:23.611741 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.611723 2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 18:41:23.623692 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.623575 2585 log.go:25] "Validated CRI v1 runtime API" Jul 09 18:41:23.626603 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.626571 2585 log.go:25] "Validated CRI v1 image API" Jul 09 18:41:23.628305 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.628286 2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 18:41:23.631109 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.631090 2585 fs.go:135] Filesystem UUIDs: map[623fbfd5-26f4-40b1-b4fb-5ca7f1200f64:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9dad348e-69e3-42ac-b8f6-a1a7d2105fd5:/dev/nvme0n1p4] Jul 09 18:41:23.631174 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.631109 2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 18:41:23.637471 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.637352 2585 manager.go:216] Machine: {Timestamp:2026-07-09 18:41:23.635923188 +0000 UTC m=+0.379906293 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102193 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22a919b32318d0771890ff38bc3679 SystemUUID:ec22a919-b323-18d0-7718-90ff38bc3679 BootID:b0df70d3-bf65-470f-9cd9-28260a5553e5 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8b:42:68:72:2d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8b:42:68:72:2d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:2f:79:b3:59:2e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 18:41:23.637471 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.637459 2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 18:41:23.637638 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.637542 2585 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 18:41:23.640255 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.640237 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 18:41:23.641453 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.641428 2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 18:41:23.641607 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.641457 2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-175.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 18:41:23.642378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.642365 2585 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 18:41:23.642378 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.642380 2585 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 18:41:23.642450 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.642387 2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 18:41:23.643258 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.643247 2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 18:41:23.645081 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.645071 2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 18:41:23.649690 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.649678 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 09 18:41:23.649815 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.649805 2585 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 18:41:23.649864 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.649836 2585 kubelet.go:486] "Attempting to sync node with API server" Jul 09 18:41:23.649864 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.649846 2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 18:41:23.649864 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.649860 2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 18:41:23.649981 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.649869 2585 kubelet.go:398] "Adding apiserver pod source" Jul 09 18:41:23.649981 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.649878 2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 18:41:23.650216 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.650205 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 18:41:23.650260 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.650235 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 18:41:23.652660 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.652643 2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 09 18:41:23.654920 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.654906 2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 18:41:23.654990 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.654937 2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 18:41:23.655090 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.655080 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 18:41:23.655120 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.655095 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 18:41:23.655120 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.655102 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 18:41:23.655120 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.655108 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 18:41:23.655120 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.655116 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 18:41:23.655120 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.655121 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 18:41:23.655783 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.655772 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 18:41:23.655783 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.655782 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 18:41:23.655857 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.655798 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 18:41:23.655857 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.655804 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 18:41:23.655857 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.655816 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 18:41:23.655857 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.655823 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 18:41:23.657519 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.657508 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 18:41:23.657519 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.657518 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 18:41:23.660681 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.660668 2585 server.go:1268] "Started kubelet" Jul 09 18:41:23.660980 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.660920 2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 18:41:23.661513 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.661208 2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 18:41:23.661646 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.661628 2585 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 18:41:23.661756 ip-10-0-143-175 systemd[1]: Started Kubernetes Kubelet. Jul 09 18:41:23.662465 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.662240 2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-175.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 18:41:23.662465 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.662239 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 18:41:23.663094 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.663003 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-175.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 18:41:23.667782 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.667756 2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 18:41:23.668793 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.668774 2585 server.go:310] "Adding debug handlers to kubelet server" Jul 09 18:41:23.668872 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.668814 2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 18:41:23.674289 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.673166 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-175.ec2.internal.18c0b332e7fe19d6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-175.ec2.internal,UID:ip-10-0-143-175.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-175.ec2.internal,},FirstTimestamp:2026-07-09 18:41:23.660642774 +0000 UTC m=+0.404625879,LastTimestamp:2026-07-09 18:41:23.660642774 +0000 UTC m=+0.404625879,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-175.ec2.internal,}" Jul 09 18:41:23.674572 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.674558 2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 18:41:23.674643 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.674563 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 18:41:23.674689 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.674645 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Jul 09 18:41:23.675544 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.675485 2585 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 18:41:23.675631 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.675550 2585 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 18:41:23.675694 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.675536 2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 18:41:23.675743 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.675722 2585 reconstruct.go:97] "Volume reconstruction finished" Jul 09 18:41:23.675743 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.675730 2585 reconciler.go:29] "Reconciler: start to sync state" Jul 09 18:41:23.675860 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.675846 2585 factory.go:55] Registering systemd factory Jul 09 18:41:23.675928 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.675864 2585 factory.go:223] Registration of the systemd container factory successfully Jul 09 18:41:23.676091 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.676070 2585 factory.go:153] Registering CRI-O factory Jul 09 18:41:23.676091 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.676085 2585 factory.go:223] Registration of the crio container factory successfully Jul 09 18:41:23.676247 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.676135 2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 18:41:23.676247 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.676155 2585 factory.go:103] Registering Raw factory Jul 09 18:41:23.676247 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.676166 2585 manager.go:1196] Started watching for new ooms in manager Jul 09 18:41:23.676605 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.676574 2585 manager.go:349] Starting recovery of all containers Jul 09 18:41:23.680903 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.680873 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 18:41:23.681040 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.681001 2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-175.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 18:41:23.681318 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.681159 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f88x5" Jul 09 18:41:23.687093 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.687074 2585 manager.go:354] Recovery completed Jul 09 18:41:23.691570 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.691558 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:41:23.692129 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.692113 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f88x5" Jul 09 18:41:23.693517 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.693498 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:41:23.693567 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.693536 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:41:23.693567 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.693554 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:41:23.694134 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.694122 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:41:23.694207 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.694137 2585 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 18:41:23.694207 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.694146 2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 18:41:23.694207 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.694162 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 09 18:41:23.694803 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.694787 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:41:23.694879 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.694819 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:41:23.694879 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.694835 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:41:23.695770 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.695707 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-175.ec2.internal.18c0b332e9f3bda2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-175.ec2.internal,UID:ip-10-0-143-175.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-175.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-175.ec2.internal,},FirstTimestamp:2026-07-09 18:41:23.693518242 +0000 UTC m=+0.437501352,LastTimestamp:2026-07-09 18:41:23.693518242 +0000 UTC m=+0.437501352,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-175.ec2.internal,}" Jul 09 18:41:23.696996 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.696984 2585 policy_none.go:49] "None policy: Start" Jul 09 18:41:23.697050 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.697000 2585 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 18:41:23.697050 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.697011 2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 18:41:23.698987 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.698975 2585 policy_none.go:47] "Start" Jul 09 18:41:23.746241 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.746221 2585 manager.go:337] "Starting Device Plugin manager" Jul 09 18:41:23.762986 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.746263 2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 18:41:23.762986 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.746276 2585 server.go:87] "Starting device plugin registration server" Jul 09 18:41:23.762986 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.746543 2585 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 18:41:23.762986 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.746559 2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 18:41:23.762986 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.746677 2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 18:41:23.762986 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.746752 2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 18:41:23.762986 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.746762 2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 18:41:23.762986 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.747344 2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 18:41:23.762986 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.747383 2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-175.ec2.internal\" not found" Jul 09 18:41:23.834884 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.834805 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 18:41:23.836144 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.836124 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 18:41:23.836209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.836177 2585 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 18:41:23.836209 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.836203 2585 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 18:41:23.836297 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.836245 2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 18:41:23.839199 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.839173 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:41:23.846827 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.846811 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:41:23.847801 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.847786 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:41:23.847877 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.847815 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:41:23.847877 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.847830 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:41:23.847877 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.847859 2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-175.ec2.internal" Jul 09 18:41:23.858871 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.858856 2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-175.ec2.internal" Jul 09 18:41:23.858949 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.858879 2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-175.ec2.internal\": node \"ip-10-0-143-175.ec2.internal\" not found" Jul 09 18:41:23.880893 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.880873 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Jul 09 18:41:23.937312 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.937263 2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal"] Jul 09 18:41:23.937432 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.937347 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:41:23.938274 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.938258 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:41:23.938374 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.938287 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:41:23.938374 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.938297 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:41:23.940714 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.940701 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:41:23.940873 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.940858 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Jul 09 18:41:23.940924 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.940889 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:41:23.941402 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.941387 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:41:23.941485 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.941415 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:41:23.941485 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.941429 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:41:23.941579 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.941560 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:41:23.941579 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.941579 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:41:23.941697 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.941612 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:41:23.943814 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.943799 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" Jul 09 18:41:23.943945 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.943822 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:41:23.944447 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.944432 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:41:23.944519 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.944453 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:41:23.944519 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.944463 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:41:23.969094 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.969075 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-175.ec2.internal\" not found" node="ip-10-0-143-175.ec2.internal" Jul 09 18:41:23.973531 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.973517 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-175.ec2.internal\" not found" node="ip-10-0-143-175.ec2.internal" Jul 09 18:41:23.977738 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.977720 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f1cebde9eeebb426c6f5a392a68065db-config\") pod \"kube-apiserver-proxy-ip-10-0-143-175.ec2.internal\" (UID: \"f1cebde9eeebb426c6f5a392a68065db\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" Jul 09 18:41:23.977813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.977753 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/53292fd96f441b9b041e7aa978696a19-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal\" (UID: \"53292fd96f441b9b041e7aa978696a19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Jul 09 18:41:23.977813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:23.977779 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53292fd96f441b9b041e7aa978696a19-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal\" (UID: \"53292fd96f441b9b041e7aa978696a19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Jul 09 18:41:23.981822 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:23.981806 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Jul 09 18:41:24.078572 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.078537 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f1cebde9eeebb426c6f5a392a68065db-config\") pod \"kube-apiserver-proxy-ip-10-0-143-175.ec2.internal\" (UID: \"f1cebde9eeebb426c6f5a392a68065db\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" Jul 09 18:41:24.078572 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.078568 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/53292fd96f441b9b041e7aa978696a19-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal\" (UID: \"53292fd96f441b9b041e7aa978696a19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Jul 09 18:41:24.078802 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.078607 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53292fd96f441b9b041e7aa978696a19-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal\" (UID: \"53292fd96f441b9b041e7aa978696a19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Jul 09 18:41:24.078802 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.078645 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53292fd96f441b9b041e7aa978696a19-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal\" (UID: \"53292fd96f441b9b041e7aa978696a19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Jul 09 18:41:24.078802 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.078657 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/53292fd96f441b9b041e7aa978696a19-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal\" (UID: \"53292fd96f441b9b041e7aa978696a19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Jul 09 18:41:24.078802 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.078655 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f1cebde9eeebb426c6f5a392a68065db-config\") pod \"kube-apiserver-proxy-ip-10-0-143-175.ec2.internal\" (UID: \"f1cebde9eeebb426c6f5a392a68065db\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" Jul 09 18:41:24.082653 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.082637 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Jul 09 18:41:24.183535 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.183468 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Jul 09 18:41:24.270696 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.270666 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Jul 09 18:41:24.276823 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.276799 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" Jul 09 18:41:24.284484 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.284465 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Jul 09 18:41:24.385032 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.384998 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Jul 09 18:41:24.485636 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.485551 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Jul 09 18:41:24.561489 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.561458 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:41:24.582482 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.582459 2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:41:24.606435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.606411 2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 18:41:24.607087 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.606532 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 18:41:24.607087 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.606557 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 18:41:24.607087 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.606564 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 18:41:24.650719 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.650688 2585 apiserver.go:52] "Watching apiserver" Jul 09 18:41:24.666461 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.666442 2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 18:41:24.667376 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.667354 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-22w7k","openshift-network-operator/iptables-alerter-mwvlh","openshift-ovn-kubernetes/ovnkube-node-gzm9x","kube-system/global-pull-secret-syncer-8mts9","kube-system/konnectivity-agent-pdlv2","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t","openshift-multus/network-metrics-daemon-qqtf8","openshift-cluster-node-tuning-operator/tuned-xqmz8","openshift-dns/node-resolver-wb72f","openshift-image-registry/node-ca-vswpc","openshift-multus/multus-additional-cni-plugins-p6dnm","openshift-multus/multus-v2k5p"] Jul 09 18:41:24.669920 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.669896 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:24.670011 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.669962 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:24.671997 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.671979 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mwvlh" Jul 09 18:41:24.675542 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.674436 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xdsq6\"" Jul 09 18:41:24.675542 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.674732 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 18:41:24.675542 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.674852 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Jul 09 18:41:24.676174 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.676156 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 18:41:24.676439 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.676421 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 18:41:24.676642 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.676621 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 18:41:24.676882 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.676867 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.678018 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.677998 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:24.678116 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.678082 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:24.679150 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.679132 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 18:41:24.679246 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.679189 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 18:41:24.679398 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.679381 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jgnzc\"" Jul 09 18:41:24.679481 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.679473 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 18:41:24.680182 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.680152 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pdlv2" Jul 09 18:41:24.680266 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.680224 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 18:41:24.680320 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.680297 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 18:41:24.680448 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.680421 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 18:41:24.682212 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682195 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-slash\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.682302 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682238 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-node-log\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.682302 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682272 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-env-overrides\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.682419 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682298 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fbaa85e6-6967-47f5-b3d7-d6ee9f54876a-iptables-alerter-script\") pod \"iptables-alerter-mwvlh\" (UID: \"fbaa85e6-6967-47f5-b3d7-d6ee9f54876a\") " pod="openshift-network-operator/iptables-alerter-mwvlh" Jul 09 18:41:24.682419 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682321 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-kubelet\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.682419 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682337 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-etc-openvswitch\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.682419 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682356 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-run-ovn\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.682419 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682378 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-592xc\"" Jul 09 18:41:24.682419 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682405 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcxbs\" (UniqueName: \"kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs\") pod \"network-check-target-22w7k\" (UID: \"eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6\") " pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:24.682711 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682434 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-cni-bin\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.682711 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682509 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-ovnkube-config\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.682711 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682521 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.682711 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682535 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-run-netns\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.682711 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682570 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-run-ovn-kubernetes\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.682711 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682609 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-ovnkube-script-lib\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.682711 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682630 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fda064a0-8a88-462c-9891-47a412d9c0f4-dbus\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:24.682711 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682662 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-ovn-node-metrics-cert\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.682978 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682724 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbaa85e6-6967-47f5-b3d7-d6ee9f54876a-host-slash\") pod \"iptables-alerter-mwvlh\" (UID: \"fbaa85e6-6967-47f5-b3d7-d6ee9f54876a\") " pod="openshift-network-operator/iptables-alerter-mwvlh" Jul 09 18:41:24.682978 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682755 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-run-systemd\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.682978 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682829 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.682978 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682885 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlnbb\" (UniqueName: \"kubernetes.io/projected/fbaa85e6-6967-47f5-b3d7-d6ee9f54876a-kube-api-access-nlnbb\") pod \"iptables-alerter-mwvlh\" (UID: \"fbaa85e6-6967-47f5-b3d7-d6ee9f54876a\") " pod="openshift-network-operator/iptables-alerter-mwvlh" Jul 09 18:41:24.682978 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682903 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fda064a0-8a88-462c-9891-47a412d9c0f4-kubelet-config\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:24.682978 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682954 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:24.682978 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.682969 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-run-openvswitch\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.683207 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.683005 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-var-lib-openvswitch\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.683207 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.683049 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-log-socket\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.683207 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.683066 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-cni-netd\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.683207 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.683115 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 18:41:24.683207 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.683116 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t6rq\" (UniqueName: \"kubernetes.io/projected/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-kube-api-access-6t6rq\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.683207 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.683185 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-systemd-units\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.683381 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.683220 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 18:41:24.684738 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.684721 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 18:41:24.684828 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.684789 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:24.684905 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.684886 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:24.685017 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.685002 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 18:41:24.685105 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.685002 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 18:41:24.685144 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.685126 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qckbz\"" Jul 09 18:41:24.687014 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.686991 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 18:41:24.687150 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.687135 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.689052 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.689037 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 18:41:24.689139 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.689128 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" Jul 09 18:41:24.689398 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.689320 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 18:41:24.689398 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.689377 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 18:41:24.689543 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.689379 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wb72f" Jul 09 18:41:24.689633 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.689386 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-g2ltf\"" Jul 09 18:41:24.691748 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.691731 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 18:41:24.691852 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.691773 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vswpc" Jul 09 18:41:24.691990 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.691969 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zrm68\"" Jul 09 18:41:24.692134 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.692120 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 18:41:24.693983 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.693963 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 18:41:24.694062 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.694000 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 18:41:24.694062 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.694054 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 18:41:24.694178 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.694118 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vjzxm\"" Jul 09 18:41:24.694329 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.694315 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.696459 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.696403 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 18:36:24 +0000 UTC" deadline="2028-02-10 13:08:03.988961861 +0000 UTC" Jul 09 18:41:24.696459 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.696456 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13938h26m39.292508713s" Jul 09 18:41:24.696634 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.696471 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7lqqx\"" Jul 09 18:41:24.696711 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.696697 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal"] Jul 09 18:41:24.696824 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.696809 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.696887 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.696847 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 18:41:24.696887 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.696878 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 18:41:24.696984 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.696911 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 18:41:24.697032 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.696983 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 18:41:24.697111 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.697093 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 18:41:24.698815 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.698793 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 18:41:24.699017 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.699003 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal"] Jul 09 18:41:24.699624 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.699612 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 18:41:24.699663 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.699629 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4phjb\"" Jul 09 18:41:24.712838 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.712822 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dsjkz" Jul 09 18:41:24.719316 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.719290 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dsjkz" Jul 09 18:41:24.776520 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.776490 2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 18:41:24.784017 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.783994 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbaa85e6-6967-47f5-b3d7-d6ee9f54876a-host-slash\") pod \"iptables-alerter-mwvlh\" (UID: \"fbaa85e6-6967-47f5-b3d7-d6ee9f54876a\") " pod="openshift-network-operator/iptables-alerter-mwvlh" Jul 09 18:41:24.784166 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784030 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.784166 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784052 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-sys\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.784166 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784102 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.784166 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784126 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-host\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.784166 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784124 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbaa85e6-6967-47f5-b3d7-d6ee9f54876a-host-slash\") pod \"iptables-alerter-mwvlh\" (UID: \"fbaa85e6-6967-47f5-b3d7-d6ee9f54876a\") " pod="openshift-network-operator/iptables-alerter-mwvlh" Jul 09 18:41:24.784166 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784145 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/397a8531-67a9-4a3f-ad6b-f450a14856a3-hosts-file\") pod \"node-resolver-wb72f\" (UID: \"397a8531-67a9-4a3f-ad6b-f450a14856a3\") " pod="openshift-dns/node-resolver-wb72f" Jul 09 18:41:24.784348 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784199 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-cnibin\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.784348 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784239 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:24.784348 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784281 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-cnibin\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.784348 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784325 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-registration-dir\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.784502 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784355 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-sysctl-conf\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.784502 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.784368 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:41:24.784502 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784378 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-etc-kubernetes\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.784502 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784408 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-run-openvswitch\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.784502 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.784432 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret podName:fda064a0-8a88-462c-9891-47a412d9c0f4 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:25.284408905 +0000 UTC m=+2.028392010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret") pod "global-pull-secret-syncer-8mts9" (UID: "fda064a0-8a88-462c-9891-47a412d9c0f4") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:41:24.784502 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784452 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-run-openvswitch\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.784502 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784474 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.784840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784508 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-multus-cni-dir\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.784840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784541 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-sys-fs\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.784840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784571 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t6rq\" (UniqueName: \"kubernetes.io/projected/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-kube-api-access-6t6rq\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.784840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784641 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs\") pod \"network-metrics-daemon-qqtf8\" (UID: \"52054539-243c-47f9-97f9-a7ff089dea64\") " pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:24.784840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784668 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/397a8531-67a9-4a3f-ad6b-f450a14856a3-tmp-dir\") pod \"node-resolver-wb72f\" (UID: \"397a8531-67a9-4a3f-ad6b-f450a14856a3\") " pod="openshift-dns/node-resolver-wb72f" Jul 09 18:41:24.784840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784709 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-var-lib-kubelet\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.784840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784740 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a21694d7-e996-4434-ba98-4dbfed837bd2-serviceca\") pod \"node-ca-vswpc\" (UID: \"a21694d7-e996-4434-ba98-4dbfed837bd2\") " pod="openshift-image-registry/node-ca-vswpc" Jul 09 18:41:24.784840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784776 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-run-netns\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.784840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784797 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-multus-daemon-config\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.784840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784816 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-kubelet\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784846 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-kubelet\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784855 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-etc-openvswitch\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784880 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-run-ovn\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784898 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr28g\" (UniqueName: \"kubernetes.io/projected/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-kube-api-access-cr28g\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784926 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-etc-openvswitch\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784931 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-lib-modules\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784941 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-run-ovn\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784947 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-tuned\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784975 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m94l\" (UniqueName: \"kubernetes.io/projected/efe9ecba-bb86-42f5-97ed-85afa0bb5969-kube-api-access-7m94l\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.784996 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-var-lib-cni-bin\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785016 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stkzr\" (UniqueName: \"kubernetes.io/projected/a21694d7-e996-4434-ba98-4dbfed837bd2-kube-api-access-stkzr\") pod \"node-ca-vswpc\" (UID: \"a21694d7-e996-4434-ba98-4dbfed837bd2\") " pod="openshift-image-registry/node-ca-vswpc" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785032 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c472afa5-1c0f-4663-9d81-12f9050de2a7-konnectivity-ca\") pod \"konnectivity-agent-pdlv2\" (UID: \"c472afa5-1c0f-4663-9d81-12f9050de2a7\") " pod="kube-system/konnectivity-agent-pdlv2" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785064 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-hostroot\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785080 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-run-netns\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785095 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fda064a0-8a88-462c-9891-47a412d9c0f4-dbus\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785109 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-socket-dir\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.785149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785124 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-cni-binary-copy\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785140 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-run-multus-certs\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785167 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-run-systemd\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785200 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-run-systemd\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785234 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-run-netns\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785252 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-multus-socket-dir-parent\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785282 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-var-lib-cni-multus\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785299 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fda064a0-8a88-462c-9891-47a412d9c0f4-dbus\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785305 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-multus-conf-dir\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785327 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlnbb\" (UniqueName: \"kubernetes.io/projected/fbaa85e6-6967-47f5-b3d7-d6ee9f54876a-kube-api-access-nlnbb\") pod \"iptables-alerter-mwvlh\" (UID: \"fbaa85e6-6967-47f5-b3d7-d6ee9f54876a\") " pod="openshift-network-operator/iptables-alerter-mwvlh" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785353 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fda064a0-8a88-462c-9891-47a412d9c0f4-kubelet-config\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785378 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-kubernetes\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785414 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-systemd\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785438 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-282bl\" (UniqueName: \"kubernetes.io/projected/397a8531-67a9-4a3f-ad6b-f450a14856a3-kube-api-access-282bl\") pod \"node-resolver-wb72f\" (UID: \"397a8531-67a9-4a3f-ad6b-f450a14856a3\") " pod="openshift-dns/node-resolver-wb72f" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785443 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fda064a0-8a88-462c-9891-47a412d9c0f4-kubelet-config\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785460 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cpbs\" (UniqueName: \"kubernetes.io/projected/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-kube-api-access-5cpbs\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785481 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-run-k8s-cni-cncf-io\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.785901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785502 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-sysconfig\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785523 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-sysctl-d\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785552 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-var-lib-openvswitch\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785579 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-log-socket\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785636 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-var-lib-openvswitch\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785682 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-cni-netd\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785709 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-cni-netd\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785711 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bk25\" (UniqueName: \"kubernetes.io/projected/52054539-243c-47f9-97f9-a7ff089dea64-kube-api-access-6bk25\") pod \"network-metrics-daemon-qqtf8\" (UID: \"52054539-243c-47f9-97f9-a7ff089dea64\") " pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785686 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-log-socket\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785737 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785761 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785784 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdjgj\" (UniqueName: \"kubernetes.io/projected/cd758b0d-acc8-4ca7-8ff1-672c3572c858-kube-api-access-rdjgj\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785810 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-systemd-units\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785837 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-slash\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785861 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-node-log\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785885 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-env-overrides\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785898 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-systemd-units\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.786613 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785903 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-slash\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785907 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-os-release\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785932 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-cni-binary-copy\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785933 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-node-log\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.785949 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-device-dir\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786046 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fbaa85e6-6967-47f5-b3d7-d6ee9f54876a-iptables-alerter-script\") pod \"iptables-alerter-mwvlh\" (UID: \"fbaa85e6-6967-47f5-b3d7-d6ee9f54876a\") " pod="openshift-network-operator/iptables-alerter-mwvlh" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786088 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-run\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786105 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-os-release\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786145 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcxbs\" (UniqueName: \"kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs\") pod \"network-check-target-22w7k\" (UID: \"eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6\") " pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786173 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-cni-bin\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786282 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-cni-bin\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786309 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-env-overrides\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786327 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-ovnkube-config\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786360 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786387 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-var-lib-kubelet\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786473 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fbaa85e6-6967-47f5-b3d7-d6ee9f54876a-iptables-alerter-script\") pod \"iptables-alerter-mwvlh\" (UID: \"fbaa85e6-6967-47f5-b3d7-d6ee9f54876a\") " pod="openshift-network-operator/iptables-alerter-mwvlh" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786477 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efe9ecba-bb86-42f5-97ed-85afa0bb5969-tmp\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.787248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786524 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-system-cni-dir\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.787742 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786544 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-run-ovn-kubernetes\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.787742 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786560 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-ovnkube-script-lib\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.787742 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786578 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a21694d7-e996-4434-ba98-4dbfed837bd2-host\") pod \"node-ca-vswpc\" (UID: \"a21694d7-e996-4434-ba98-4dbfed837bd2\") " pod="openshift-image-registry/node-ca-vswpc" Jul 09 18:41:24.787742 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786613 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-system-cni-dir\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.787742 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786635 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c472afa5-1c0f-4663-9d81-12f9050de2a7-agent-certs\") pod \"konnectivity-agent-pdlv2\" (UID: \"c472afa5-1c0f-4663-9d81-12f9050de2a7\") " pod="kube-system/konnectivity-agent-pdlv2" Jul 09 18:41:24.787742 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786626 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-host-run-ovn-kubernetes\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.787742 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786678 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-etc-selinux\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.787742 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786700 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-modprobe-d\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.787742 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786757 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-ovn-node-metrics-cert\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.787742 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.786898 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-ovnkube-config\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.787742 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.787048 2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 18:41:24.787742 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.787567 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-ovnkube-script-lib\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.790693 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.790675 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-ovn-node-metrics-cert\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.796476 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.796431 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t6rq\" (UniqueName: \"kubernetes.io/projected/5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985-kube-api-access-6t6rq\") pod \"ovnkube-node-gzm9x\" (UID: \"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:24.797033 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.797018 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:41:24.797086 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.797038 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:41:24.797086 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.797047 2585 projected.go:196] Error preparing data for projected volume kube-api-access-rcxbs for pod openshift-network-diagnostics/network-check-target-22w7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:41:24.797164 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.797153 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs podName:eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:25.297135548 +0000 UTC m=+2.041118657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rcxbs" (UniqueName: "kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs") pod "network-check-target-22w7k" (UID: "eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:41:24.799039 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.799018 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlnbb\" (UniqueName: \"kubernetes.io/projected/fbaa85e6-6967-47f5-b3d7-d6ee9f54876a-kube-api-access-nlnbb\") pod \"iptables-alerter-mwvlh\" (UID: \"fbaa85e6-6967-47f5-b3d7-d6ee9f54876a\") " pod="openshift-network-operator/iptables-alerter-mwvlh" Jul 09 18:41:24.853722 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.853695 2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:41:24.887711 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887662 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-cnibin\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.887711 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887702 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-cnibin\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.887899 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887719 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-registration-dir\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.887899 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887751 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-sysctl-conf\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.887899 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887769 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-etc-kubernetes\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.887899 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887793 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-cnibin\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.887899 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887800 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.887899 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887817 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-multus-cni-dir\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.887899 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887824 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-registration-dir\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.887899 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887830 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-etc-kubernetes\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.887899 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887832 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-sys-fs\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.887899 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887875 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-multus-cni-dir\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.887899 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887895 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs\") pod \"network-metrics-daemon-qqtf8\" (UID: \"52054539-243c-47f9-97f9-a7ff089dea64\") " pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:24.887899 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887792 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-cnibin\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887903 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-sysctl-conf\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887927 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-sys-fs\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.887924 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/397a8531-67a9-4a3f-ad6b-f450a14856a3-tmp-dir\") pod \"node-resolver-wb72f\" (UID: \"397a8531-67a9-4a3f-ad6b-f450a14856a3\") " pod="openshift-dns/node-resolver-wb72f" Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.887981 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888039 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-var-lib-kubelet\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:24.888051 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs podName:52054539-243c-47f9-97f9-a7ff089dea64 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:25.388034152 +0000 UTC m=+2.132017247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs") pod "network-metrics-daemon-qqtf8" (UID: "52054539-243c-47f9-97f9-a7ff089dea64") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888079 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a21694d7-e996-4434-ba98-4dbfed837bd2-serviceca\") pod \"node-ca-vswpc\" (UID: \"a21694d7-e996-4434-ba98-4dbfed837bd2\") " pod="openshift-image-registry/node-ca-vswpc" Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888102 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-run-netns\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888124 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-multus-daemon-config\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888124 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-var-lib-kubelet\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888152 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr28g\" (UniqueName: \"kubernetes.io/projected/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-kube-api-access-cr28g\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888171 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-run-netns\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888179 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-lib-modules\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888201 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-tuned\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888205 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/397a8531-67a9-4a3f-ad6b-f450a14856a3-tmp-dir\") pod \"node-resolver-wb72f\" (UID: \"397a8531-67a9-4a3f-ad6b-f450a14856a3\") " pod="openshift-dns/node-resolver-wb72f" Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888222 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m94l\" (UniqueName: \"kubernetes.io/projected/efe9ecba-bb86-42f5-97ed-85afa0bb5969-kube-api-access-7m94l\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.888460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888247 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-var-lib-cni-bin\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888268 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stkzr\" (UniqueName: \"kubernetes.io/projected/a21694d7-e996-4434-ba98-4dbfed837bd2-kube-api-access-stkzr\") pod \"node-ca-vswpc\" (UID: \"a21694d7-e996-4434-ba98-4dbfed837bd2\") " pod="openshift-image-registry/node-ca-vswpc" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888290 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c472afa5-1c0f-4663-9d81-12f9050de2a7-konnectivity-ca\") pod \"konnectivity-agent-pdlv2\" (UID: \"c472afa5-1c0f-4663-9d81-12f9050de2a7\") " pod="kube-system/konnectivity-agent-pdlv2" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888311 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-hostroot\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888335 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-socket-dir\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888334 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-lib-modules\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888360 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-cni-binary-copy\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888398 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888454 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-var-lib-cni-bin\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888499 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-hostroot\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888553 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a21694d7-e996-4434-ba98-4dbfed837bd2-serviceca\") pod \"node-ca-vswpc\" (UID: \"a21694d7-e996-4434-ba98-4dbfed837bd2\") " pod="openshift-image-registry/node-ca-vswpc" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888610 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-run-multus-certs\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888629 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-socket-dir\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888641 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-multus-socket-dir-parent\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888672 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-var-lib-cni-multus\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888680 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-multus-daemon-config\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888688 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-multus-socket-dir-parent\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888715 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-run-multus-certs\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.889503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888731 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-multus-conf-dir\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888735 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-var-lib-cni-multus\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888783 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-multus-conf-dir\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888823 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-kubernetes\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888846 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-systemd\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888871 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-282bl\" (UniqueName: \"kubernetes.io/projected/397a8531-67a9-4a3f-ad6b-f450a14856a3-kube-api-access-282bl\") pod \"node-resolver-wb72f\" (UID: \"397a8531-67a9-4a3f-ad6b-f450a14856a3\") " pod="openshift-dns/node-resolver-wb72f" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888868 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-kubernetes\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888886 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-systemd\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888895 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cpbs\" (UniqueName: \"kubernetes.io/projected/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-kube-api-access-5cpbs\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888914 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-cni-binary-copy\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888944 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-run-k8s-cni-cncf-io\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888970 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-sysconfig\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888994 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-sysctl-d\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.888997 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-host-run-k8s-cni-cncf-io\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889035 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c472afa5-1c0f-4663-9d81-12f9050de2a7-konnectivity-ca\") pod \"konnectivity-agent-pdlv2\" (UID: \"c472afa5-1c0f-4663-9d81-12f9050de2a7\") " pod="kube-system/konnectivity-agent-pdlv2" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889043 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-sysconfig\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889026 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bk25\" (UniqueName: \"kubernetes.io/projected/52054539-243c-47f9-97f9-a7ff089dea64-kube-api-access-6bk25\") pod \"network-metrics-daemon-qqtf8\" (UID: \"52054539-243c-47f9-97f9-a7ff089dea64\") " pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889089 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.890435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889111 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-sysctl-d\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889116 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889142 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdjgj\" (UniqueName: \"kubernetes.io/projected/cd758b0d-acc8-4ca7-8ff1-672c3572c858-kube-api-access-rdjgj\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889170 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-os-release\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889176 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889196 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-cni-binary-copy\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889220 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-device-dir\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889238 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-os-release\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889244 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-run\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889267 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-os-release\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889268 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889319 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-device-dir\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889323 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889343 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-run\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889373 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-var-lib-kubelet\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889396 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efe9ecba-bb86-42f5-97ed-85afa0bb5969-tmp\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889414 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-system-cni-dir\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.891193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889422 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-var-lib-kubelet\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889403 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-os-release\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889474 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-system-cni-dir\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889476 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a21694d7-e996-4434-ba98-4dbfed837bd2-host\") pod \"node-ca-vswpc\" (UID: \"a21694d7-e996-4434-ba98-4dbfed837bd2\") " pod="openshift-image-registry/node-ca-vswpc" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889519 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-system-cni-dir\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889544 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c472afa5-1c0f-4663-9d81-12f9050de2a7-agent-certs\") pod \"konnectivity-agent-pdlv2\" (UID: \"c472afa5-1c0f-4663-9d81-12f9050de2a7\") " pod="kube-system/konnectivity-agent-pdlv2" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889622 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-system-cni-dir\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889607 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a21694d7-e996-4434-ba98-4dbfed837bd2-host\") pod \"node-ca-vswpc\" (UID: \"a21694d7-e996-4434-ba98-4dbfed837bd2\") " pod="openshift-image-registry/node-ca-vswpc" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889639 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-etc-selinux\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889730 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cd758b0d-acc8-4ca7-8ff1-672c3572c858-etc-selinux\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889750 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-modprobe-d\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889768 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-cni-binary-copy\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889778 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-sys\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889801 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-host\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889824 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/397a8531-67a9-4a3f-ad6b-f450a14856a3-hosts-file\") pod \"node-resolver-wb72f\" (UID: \"397a8531-67a9-4a3f-ad6b-f450a14856a3\") " pod="openshift-dns/node-resolver-wb72f" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889829 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889835 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-sys\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889882 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-host\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.891813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.889931 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/397a8531-67a9-4a3f-ad6b-f450a14856a3-hosts-file\") pod \"node-resolver-wb72f\" (UID: \"397a8531-67a9-4a3f-ad6b-f450a14856a3\") " pod="openshift-dns/node-resolver-wb72f" Jul 09 18:41:24.892303 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.890112 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-modprobe-d\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.892303 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.890908 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/efe9ecba-bb86-42f5-97ed-85afa0bb5969-etc-tuned\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.892303 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.891436 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efe9ecba-bb86-42f5-97ed-85afa0bb5969-tmp\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.892608 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.892575 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c472afa5-1c0f-4663-9d81-12f9050de2a7-agent-certs\") pod \"konnectivity-agent-pdlv2\" (UID: \"c472afa5-1c0f-4663-9d81-12f9050de2a7\") " pod="kube-system/konnectivity-agent-pdlv2" Jul 09 18:41:24.898443 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.898416 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m94l\" (UniqueName: \"kubernetes.io/projected/efe9ecba-bb86-42f5-97ed-85afa0bb5969-kube-api-access-7m94l\") pod \"tuned-xqmz8\" (UID: \"efe9ecba-bb86-42f5-97ed-85afa0bb5969\") " pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:24.898925 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.898903 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stkzr\" (UniqueName: \"kubernetes.io/projected/a21694d7-e996-4434-ba98-4dbfed837bd2-kube-api-access-stkzr\") pod \"node-ca-vswpc\" (UID: \"a21694d7-e996-4434-ba98-4dbfed837bd2\") " pod="openshift-image-registry/node-ca-vswpc" Jul 09 18:41:24.899376 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.899353 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr28g\" (UniqueName: \"kubernetes.io/projected/6afc4ccb-8f0f-4246-9d5f-d873d013a3b7-kube-api-access-cr28g\") pod \"multus-additional-cni-plugins-p6dnm\" (UID: \"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7\") " pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:24.900235 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.900214 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdjgj\" (UniqueName: \"kubernetes.io/projected/cd758b0d-acc8-4ca7-8ff1-672c3572c858-kube-api-access-rdjgj\") pod \"aws-ebs-csi-driver-node-mjn7t\" (UID: \"cd758b0d-acc8-4ca7-8ff1-672c3572c858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:24.900343 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.900286 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bk25\" (UniqueName: \"kubernetes.io/projected/52054539-243c-47f9-97f9-a7ff089dea64-kube-api-access-6bk25\") pod \"network-metrics-daemon-qqtf8\" (UID: \"52054539-243c-47f9-97f9-a7ff089dea64\") " pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:24.900672 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.900656 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cpbs\" (UniqueName: \"kubernetes.io/projected/2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f-kube-api-access-5cpbs\") pod \"multus-v2k5p\" (UID: \"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f\") " pod="openshift-multus/multus-v2k5p" Jul 09 18:41:24.901080 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.901062 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-282bl\" (UniqueName: \"kubernetes.io/projected/397a8531-67a9-4a3f-ad6b-f450a14856a3-kube-api-access-282bl\") pod \"node-resolver-wb72f\" (UID: \"397a8531-67a9-4a3f-ad6b-f450a14856a3\") " pod="openshift-dns/node-resolver-wb72f" Jul 09 18:41:24.916230 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:24.916203 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53292fd96f441b9b041e7aa978696a19.slice/crio-291dca44302c4a1d4026ed05a56633589e0ea1b118ef3a88b059cf9eb748aaac WatchSource:0}: Error finding container 291dca44302c4a1d4026ed05a56633589e0ea1b118ef3a88b059cf9eb748aaac: Status 404 returned error can't find the container with id 291dca44302c4a1d4026ed05a56633589e0ea1b118ef3a88b059cf9eb748aaac Jul 09 18:41:24.921135 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.921122 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 18:41:24.996565 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:24.996514 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mwvlh" Jul 09 18:41:25.006000 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:25.005974 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbaa85e6_6967_47f5_b3d7_d6ee9f54876a.slice/crio-2f06da16fef586d3f8b66e719e6cea0c3d43d897655a3a60aac5ee2e253bc9cb WatchSource:0}: Error finding container 2f06da16fef586d3f8b66e719e6cea0c3d43d897655a3a60aac5ee2e253bc9cb: Status 404 returned error can't find the container with id 2f06da16fef586d3f8b66e719e6cea0c3d43d897655a3a60aac5ee2e253bc9cb Jul 09 18:41:25.009926 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.009911 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" Jul 09 18:41:25.017546 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:25.017524 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe9ecba_bb86_42f5_97ed_85afa0bb5969.slice/crio-cad052de816849e777b6a4ccf2b69fdcb631c01a6c03c3b6bf017df5dd07f1a6 WatchSource:0}: Error finding container cad052de816849e777b6a4ccf2b69fdcb631c01a6c03c3b6bf017df5dd07f1a6: Status 404 returned error can't find the container with id cad052de816849e777b6a4ccf2b69fdcb631c01a6c03c3b6bf017df5dd07f1a6 Jul 09 18:41:25.026868 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.026850 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:25.036183 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:25.036162 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a43cf7e_2232_4fdc_aa1d_4ac08f9e9985.slice/crio-efe45cf4a7a843b30c674e05a8a2bab2243bc6cd14b9e9ef8cd3117e1d1074da WatchSource:0}: Error finding container efe45cf4a7a843b30c674e05a8a2bab2243bc6cd14b9e9ef8cd3117e1d1074da: Status 404 returned error can't find the container with id efe45cf4a7a843b30c674e05a8a2bab2243bc6cd14b9e9ef8cd3117e1d1074da Jul 09 18:41:25.036924 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:25.036903 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1cebde9eeebb426c6f5a392a68065db.slice/crio-af550ad1b5abc95f09af20827ac37dcc1d07c60bfd84d2272e960dbedfa3e686 WatchSource:0}: Error finding container af550ad1b5abc95f09af20827ac37dcc1d07c60bfd84d2272e960dbedfa3e686: Status 404 returned error can't find the container with id af550ad1b5abc95f09af20827ac37dcc1d07c60bfd84d2272e960dbedfa3e686 Jul 09 18:41:25.046192 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.046174 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pdlv2" Jul 09 18:41:25.054615 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:25.054596 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc472afa5_1c0f_4663_9d81_12f9050de2a7.slice/crio-1bf469e7f6f26f4aaccc1dbbfb2af694d03d6bed7cb03d8a698ca3bbaa52f19f WatchSource:0}: Error finding container 1bf469e7f6f26f4aaccc1dbbfb2af694d03d6bed7cb03d8a698ca3bbaa52f19f: Status 404 returned error can't find the container with id 1bf469e7f6f26f4aaccc1dbbfb2af694d03d6bed7cb03d8a698ca3bbaa52f19f Jul 09 18:41:25.062392 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.062377 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" Jul 09 18:41:25.070299 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:25.070280 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd758b0d_acc8_4ca7_8ff1_672c3572c858.slice/crio-7e0e168c5a3194651d248bc46797dbb4168bc88c2700bdcf2c77796d7af907c5 WatchSource:0}: Error finding container 7e0e168c5a3194651d248bc46797dbb4168bc88c2700bdcf2c77796d7af907c5: Status 404 returned error can't find the container with id 7e0e168c5a3194651d248bc46797dbb4168bc88c2700bdcf2c77796d7af907c5 Jul 09 18:41:25.078959 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.078939 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wb72f" Jul 09 18:41:25.086851 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:25.086826 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod397a8531_67a9_4a3f_ad6b_f450a14856a3.slice/crio-3674928317ffe1a6263fbbf918b9327d003c58fbf9b256b2ab215f8668fe81e6 WatchSource:0}: Error finding container 3674928317ffe1a6263fbbf918b9327d003c58fbf9b256b2ab215f8668fe81e6: Status 404 returned error can't find the container with id 3674928317ffe1a6263fbbf918b9327d003c58fbf9b256b2ab215f8668fe81e6 Jul 09 18:41:25.094867 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.094849 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vswpc" Jul 09 18:41:25.103278 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:25.103257 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda21694d7_e996_4434_ba98_4dbfed837bd2.slice/crio-3bcc1996348436bd6efd10f0dc42cec5916525c9300094348403cfc526334aa4 WatchSource:0}: Error finding container 3bcc1996348436bd6efd10f0dc42cec5916525c9300094348403cfc526334aa4: Status 404 returned error can't find the container with id 3bcc1996348436bd6efd10f0dc42cec5916525c9300094348403cfc526334aa4 Jul 09 18:41:25.120015 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.119995 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p6dnm" Jul 09 18:41:25.121193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.121176 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v2k5p" Jul 09 18:41:25.129444 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:25.129283 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6afc4ccb_8f0f_4246_9d5f_d873d013a3b7.slice/crio-3f1f3d23350528a28b40b48e61612596ddde14acb0793bbaf9a551b50c0fc815 WatchSource:0}: Error finding container 3f1f3d23350528a28b40b48e61612596ddde14acb0793bbaf9a551b50c0fc815: Status 404 returned error can't find the container with id 3f1f3d23350528a28b40b48e61612596ddde14acb0793bbaf9a551b50c0fc815 Jul 09 18:41:25.131414 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:41:25.131395 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e1679f5_8a35_42d4_b1ea_8eb4aeefec5f.slice/crio-e9fc82fee23e39d02beb6cfa5a49f91a5507137a96dbf885866cfa3b239d3c18 WatchSource:0}: Error finding container e9fc82fee23e39d02beb6cfa5a49f91a5507137a96dbf885866cfa3b239d3c18: Status 404 returned error can't find the container with id e9fc82fee23e39d02beb6cfa5a49f91a5507137a96dbf885866cfa3b239d3c18 Jul 09 18:41:25.292514 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.292413 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:25.292699 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:25.292589 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:41:25.292699 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:25.292692 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret podName:fda064a0-8a88-462c-9891-47a412d9c0f4 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:26.292671419 +0000 UTC m=+3.036654515 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret") pod "global-pull-secret-syncer-8mts9" (UID: "fda064a0-8a88-462c-9891-47a412d9c0f4") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:41:25.393358 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.393324 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcxbs\" (UniqueName: \"kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs\") pod \"network-check-target-22w7k\" (UID: \"eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6\") " pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:25.393518 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.393389 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs\") pod \"network-metrics-daemon-qqtf8\" (UID: \"52054539-243c-47f9-97f9-a7ff089dea64\") " pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:25.393518 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:25.393491 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:41:25.393518 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:25.393495 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:41:25.393518 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:25.393519 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:41:25.393651 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:25.393529 2585 projected.go:196] Error preparing data for projected volume kube-api-access-rcxbs for pod openshift-network-diagnostics/network-check-target-22w7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:41:25.393651 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:25.393554 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs podName:52054539-243c-47f9-97f9-a7ff089dea64 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:26.393534991 +0000 UTC m=+3.137518104 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs") pod "network-metrics-daemon-qqtf8" (UID: "52054539-243c-47f9-97f9-a7ff089dea64") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:41:25.393651 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:25.393575 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs podName:eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:26.393560197 +0000 UTC m=+3.137543291 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcxbs" (UniqueName: "kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs") pod "network-check-target-22w7k" (UID: "eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:41:25.724290 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.724164 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 18:36:25 +0000 UTC" deadline="2027-12-21 18:31:20.219352459 +0000 UTC" Jul 09 18:41:25.724290 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.724239 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12719h49m54.495118383s" Jul 09 18:41:25.836688 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.836641 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:25.836899 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:25.836807 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:25.848990 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.848933 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pdlv2" event={"ID":"c472afa5-1c0f-4663-9d81-12f9050de2a7","Type":"ContainerStarted","Data":"1bf469e7f6f26f4aaccc1dbbfb2af694d03d6bed7cb03d8a698ca3bbaa52f19f"} Jul 09 18:41:25.851991 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.851960 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" event={"ID":"f1cebde9eeebb426c6f5a392a68065db","Type":"ContainerStarted","Data":"af550ad1b5abc95f09af20827ac37dcc1d07c60bfd84d2272e960dbedfa3e686"} Jul 09 18:41:25.855868 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.855843 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" event={"ID":"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985","Type":"ContainerStarted","Data":"efe45cf4a7a843b30c674e05a8a2bab2243bc6cd14b9e9ef8cd3117e1d1074da"} Jul 09 18:41:25.865214 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.865165 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" event={"ID":"efe9ecba-bb86-42f5-97ed-85afa0bb5969","Type":"ContainerStarted","Data":"cad052de816849e777b6a4ccf2b69fdcb631c01a6c03c3b6bf017df5dd07f1a6"} Jul 09 18:41:25.871023 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.870992 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6dnm" event={"ID":"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7","Type":"ContainerStarted","Data":"3f1f3d23350528a28b40b48e61612596ddde14acb0793bbaf9a551b50c0fc815"} Jul 09 18:41:25.879115 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.879085 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vswpc" event={"ID":"a21694d7-e996-4434-ba98-4dbfed837bd2","Type":"ContainerStarted","Data":"3bcc1996348436bd6efd10f0dc42cec5916525c9300094348403cfc526334aa4"} Jul 09 18:41:25.885459 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.885422 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mwvlh" event={"ID":"fbaa85e6-6967-47f5-b3d7-d6ee9f54876a","Type":"ContainerStarted","Data":"2f06da16fef586d3f8b66e719e6cea0c3d43d897655a3a60aac5ee2e253bc9cb"} Jul 09 18:41:25.887362 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.887334 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" event={"ID":"53292fd96f441b9b041e7aa978696a19","Type":"ContainerStarted","Data":"291dca44302c4a1d4026ed05a56633589e0ea1b118ef3a88b059cf9eb748aaac"} Jul 09 18:41:25.889693 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.889665 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v2k5p" event={"ID":"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f","Type":"ContainerStarted","Data":"e9fc82fee23e39d02beb6cfa5a49f91a5507137a96dbf885866cfa3b239d3c18"} Jul 09 18:41:25.899968 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.899939 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wb72f" event={"ID":"397a8531-67a9-4a3f-ad6b-f450a14856a3","Type":"ContainerStarted","Data":"3674928317ffe1a6263fbbf918b9327d003c58fbf9b256b2ab215f8668fe81e6"} Jul 09 18:41:25.903881 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.903853 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" event={"ID":"cd758b0d-acc8-4ca7-8ff1-672c3572c858","Type":"ContainerStarted","Data":"7e0e168c5a3194651d248bc46797dbb4168bc88c2700bdcf2c77796d7af907c5"} Jul 09 18:41:25.941775 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:25.941737 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:41:26.301237 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:26.301194 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:26.301491 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:26.301397 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:41:26.301575 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:26.301489 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret podName:fda064a0-8a88-462c-9891-47a412d9c0f4 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:28.301466637 +0000 UTC m=+5.045449752 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret") pod "global-pull-secret-syncer-8mts9" (UID: "fda064a0-8a88-462c-9891-47a412d9c0f4") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:41:26.402615 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:26.401659 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs\") pod \"network-metrics-daemon-qqtf8\" (UID: \"52054539-243c-47f9-97f9-a7ff089dea64\") " pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:26.402615 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:26.401764 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcxbs\" (UniqueName: \"kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs\") pod \"network-check-target-22w7k\" (UID: \"eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6\") " pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:26.402615 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:26.402002 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:41:26.402615 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:26.402027 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:41:26.402615 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:26.402041 2585 projected.go:196] Error preparing data for projected volume kube-api-access-rcxbs for pod openshift-network-diagnostics/network-check-target-22w7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:41:26.402615 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:26.402103 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs podName:eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:28.40208272 +0000 UTC m=+5.146065832 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcxbs" (UniqueName: "kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs") pod "network-check-target-22w7k" (UID: "eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:41:26.402615 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:26.402529 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:41:26.402615 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:26.402584 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs podName:52054539-243c-47f9-97f9-a7ff089dea64 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:28.402566921 +0000 UTC m=+5.146550017 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs") pod "network-metrics-daemon-qqtf8" (UID: "52054539-243c-47f9-97f9-a7ff089dea64") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:41:26.636406 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:26.636323 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:41:26.724998 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:26.724952 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 18:36:25 +0000 UTC" deadline="2027-12-15 16:00:11.821554053 +0000 UTC" Jul 09 18:41:26.724998 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:26.724997 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12573h18m45.096561312s" Jul 09 18:41:26.837156 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:26.837114 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:26.837355 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:26.837259 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:26.837738 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:26.837718 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:26.837864 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:26.837843 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:27.580018 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:27.579977 2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:41:27.839642 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:27.839609 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:27.840135 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:27.839774 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:28.318070 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:28.317319 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:28.318070 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:28.317512 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:41:28.318070 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:28.317581 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret podName:fda064a0-8a88-462c-9891-47a412d9c0f4 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:32.317558762 +0000 UTC m=+9.061541883 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret") pod "global-pull-secret-syncer-8mts9" (UID: "fda064a0-8a88-462c-9891-47a412d9c0f4") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:41:28.417998 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:28.417958 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs\") pod \"network-metrics-daemon-qqtf8\" (UID: \"52054539-243c-47f9-97f9-a7ff089dea64\") " pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:28.418171 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:28.418041 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcxbs\" (UniqueName: \"kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs\") pod \"network-check-target-22w7k\" (UID: \"eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6\") " pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:28.418253 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:28.418214 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:41:28.418253 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:28.418233 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:41:28.418253 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:28.418247 2585 projected.go:196] Error preparing data for projected volume kube-api-access-rcxbs for pod openshift-network-diagnostics/network-check-target-22w7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:41:28.418436 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:28.418307 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs podName:eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:32.418287281 +0000 UTC m=+9.162270396 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcxbs" (UniqueName: "kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs") pod "network-check-target-22w7k" (UID: "eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:41:28.418786 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:28.418765 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:41:28.418874 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:28.418824 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs podName:52054539-243c-47f9-97f9-a7ff089dea64 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:32.41880683 +0000 UTC m=+9.162789938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs") pod "network-metrics-daemon-qqtf8" (UID: "52054539-243c-47f9-97f9-a7ff089dea64") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:41:28.837120 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:28.837083 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:28.837313 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:28.837225 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:28.837716 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:28.837692 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:28.837832 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:28.837811 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:29.839405 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:29.839371 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:29.839928 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:29.839507 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:30.837764 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:30.837036 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:30.837764 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:30.837182 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:30.837764 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:30.837585 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:30.837764 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:30.837714 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:31.837120 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:31.837066 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:31.837617 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:31.837207 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:32.353499 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:32.353457 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:32.353696 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:32.353657 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:41:32.353756 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:32.353739 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret podName:fda064a0-8a88-462c-9891-47a412d9c0f4 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:40.353719277 +0000 UTC m=+17.097702374 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret") pod "global-pull-secret-syncer-8mts9" (UID: "fda064a0-8a88-462c-9891-47a412d9c0f4") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:41:32.454725 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:32.454682 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcxbs\" (UniqueName: \"kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs\") pod \"network-check-target-22w7k\" (UID: \"eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6\") " pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:32.454892 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:32.454764 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs\") pod \"network-metrics-daemon-qqtf8\" (UID: \"52054539-243c-47f9-97f9-a7ff089dea64\") " pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:32.454892 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:32.454871 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:41:32.455011 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:32.454891 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:41:32.455011 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:32.454917 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:41:32.455011 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:32.454927 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs podName:52054539-243c-47f9-97f9-a7ff089dea64 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:40.454913122 +0000 UTC m=+17.198896215 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs") pod "network-metrics-daemon-qqtf8" (UID: "52054539-243c-47f9-97f9-a7ff089dea64") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:41:32.455011 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:32.454931 2585 projected.go:196] Error preparing data for projected volume kube-api-access-rcxbs for pod openshift-network-diagnostics/network-check-target-22w7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:41:32.455011 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:32.454986 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs podName:eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:40.454966848 +0000 UTC m=+17.198949959 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcxbs" (UniqueName: "kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs") pod "network-check-target-22w7k" (UID: "eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:41:32.836763 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:32.836728 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:32.836763 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:32.836762 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:32.836963 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:32.836875 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:32.837031 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:32.837009 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:33.837479 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:33.837446 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:33.837921 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:33.837547 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:34.837441 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:34.837410 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:34.837640 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:34.837445 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:34.837640 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:34.837544 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:34.837990 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:34.837689 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:35.837259 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:35.837227 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:35.840249 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:35.837343 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:36.836983 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:36.836946 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:36.836983 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:36.836971 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:36.837236 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:36.837078 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:36.837282 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:36.837225 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:37.837125 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:37.837042 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:37.837566 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:37.837176 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:38.836893 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:38.836858 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:38.836893 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:38.836881 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:38.837111 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:38.836988 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:38.837169 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:38.837142 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:39.836958 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:39.836921 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:39.837135 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:39.837072 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:40.419123 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:40.419076 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:40.419549 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:40.419258 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:41:40.419549 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:40.419325 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret podName:fda064a0-8a88-462c-9891-47a412d9c0f4 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:56.419306569 +0000 UTC m=+33.163289665 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret") pod "global-pull-secret-syncer-8mts9" (UID: "fda064a0-8a88-462c-9891-47a412d9c0f4") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:41:40.520080 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:40.520027 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs\") pod \"network-metrics-daemon-qqtf8\" (UID: \"52054539-243c-47f9-97f9-a7ff089dea64\") " pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:40.520258 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:40.520131 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcxbs\" (UniqueName: \"kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs\") pod \"network-check-target-22w7k\" (UID: \"eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6\") " pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:40.520339 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:40.520312 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:41:40.520339 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:40.520327 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:41:40.520462 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:40.520344 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:41:40.520462 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:40.520364 2585 projected.go:196] Error preparing data for projected volume kube-api-access-rcxbs for pod openshift-network-diagnostics/network-check-target-22w7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:41:40.520462 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:40.520411 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs podName:52054539-243c-47f9-97f9-a7ff089dea64 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:56.520377361 +0000 UTC m=+33.264360474 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs") pod "network-metrics-daemon-qqtf8" (UID: "52054539-243c-47f9-97f9-a7ff089dea64") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:41:40.520462 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:40.520428 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs podName:eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6 nodeName:}" failed. No retries permitted until 2026-07-09 18:41:56.520421154 +0000 UTC m=+33.264404259 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcxbs" (UniqueName: "kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs") pod "network-check-target-22w7k" (UID: "eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:41:40.836667 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:40.836577 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:40.836869 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:40.836596 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:40.836869 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:40.836704 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:40.836974 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:40.836879 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:41.836774 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:41.836683 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:41.837221 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:41.836826 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:42.836561 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:42.836527 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:42.836765 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:42.836532 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:42.836765 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:42.836667 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:42.837160 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:42.836768 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:43.837999 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:43.837964 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:43.838464 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:43.838067 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:44.837513 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:44.837471 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:44.837703 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:44.837628 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:44.837703 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:44.837683 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:44.837825 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:44.837789 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:45.836820 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:45.836788 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:45.837352 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:45.836900 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:46.836772 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:46.836490 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:46.836956 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:46.836571 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:46.836956 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:46.836899 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:46.837993 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:46.837006 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:46.972010 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:46.971979 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" event={"ID":"f1cebde9eeebb426c6f5a392a68065db","Type":"ContainerStarted","Data":"93731db55d18857603988fd9a08d2e4bad6c6d9045cac6f0076c2c29aaa12f3e"} Jul 09 18:41:46.978710 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:46.978691 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 18:41:46.979050 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:46.979026 2585 generic.go:356] "Generic (PLEG): container finished" podID="5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985" containerID="923982b6f11b304279772271957d95e33f9ed4b5ea5a7ee043ee2eb8192ff55d" exitCode=1 Jul 09 18:41:46.979138 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:46.979093 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" event={"ID":"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985","Type":"ContainerStarted","Data":"4ccab8d0696441cf917518822951f12716a585e06ba981aa812a334164ee61a1"} Jul 09 18:41:46.979187 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:46.979174 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" event={"ID":"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985","Type":"ContainerStarted","Data":"f8991b1aa474959a9e8ce04fc6af5ddac4937844ec1492aa99437c964578d626"} Jul 09 18:41:46.979239 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:46.979225 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" event={"ID":"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985","Type":"ContainerStarted","Data":"c3b6a93ca91ed264ab7e956ea975ef78893584d303816b00f61e2b2e87d4d110"} Jul 09 18:41:46.979289 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:46.979276 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" event={"ID":"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985","Type":"ContainerDied","Data":"923982b6f11b304279772271957d95e33f9ed4b5ea5a7ee043ee2eb8192ff55d"} Jul 09 18:41:46.979342 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:46.979330 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" event={"ID":"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985","Type":"ContainerStarted","Data":"4d9772d14b11d929e946d6cf65e0f84aa5317f5ee3a239a137010f4d999325ba"} Jul 09 18:41:46.980707 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:46.980684 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" event={"ID":"efe9ecba-bb86-42f5-97ed-85afa0bb5969","Type":"ContainerStarted","Data":"ee1d9d3962b6fd1865e072bbf6e564cdc3991195bdf4be223d8406d8587b6b28"} Jul 09 18:41:46.983138 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:46.983111 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v2k5p" event={"ID":"2e1679f5-8a35-42d4-b1ea-8eb4aeefec5f","Type":"ContainerStarted","Data":"fb1924e57b2c1e53b4ac337a38726661cc83eb079b2cdd686154080204a9e649"} Jul 09 18:41:46.988219 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:46.988178 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" podStartSLOduration=21.988166342 podStartE2EDuration="21.988166342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:41:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:41:46.987541376 +0000 UTC m=+23.731524490" watchObservedRunningTime="2026-07-09 18:41:46.988166342 +0000 UTC m=+23.732149456" Jul 09 18:41:47.005542 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:47.005370 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xqmz8" podStartSLOduration=1.967514994 podStartE2EDuration="23.005351866s" totalImagesPullingTime="21.037836872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:41:47.00436909 +0000 UTC m=+23.748352195" watchObservedRunningTime="2026-07-09 18:41:47.005351866 +0000 UTC m=+23.749334981" Jul 09 18:41:47.031219 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:47.031064 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-v2k5p" podStartSLOduration=2.070708895 podStartE2EDuration="23.031050852s" totalImagesPullingTime="20.960341957s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:41:47.03095087 +0000 UTC m=+23.774933983" watchObservedRunningTime="2026-07-09 18:41:47.031050852 +0000 UTC m=+23.775033966" Jul 09 18:41:47.836898 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:47.836831 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:47.837048 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:47.836918 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:47.987566 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:47.987527 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wb72f" event={"ID":"397a8531-67a9-4a3f-ad6b-f450a14856a3","Type":"ContainerStarted","Data":"cdc663789e727b9d5c34ca3d8797eb6c6a58684a05aed9f7e03db57cd73c1fd9"} Jul 09 18:41:47.989231 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:47.989200 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" event={"ID":"cd758b0d-acc8-4ca7-8ff1-672c3572c858","Type":"ContainerStarted","Data":"dd41856c3de01c8b7cba5c92879d48efe16bb0792301b551439d1f6a12a2ea69"} Jul 09 18:41:47.990948 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:47.990917 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pdlv2" event={"ID":"c472afa5-1c0f-4663-9d81-12f9050de2a7","Type":"ContainerStarted","Data":"aee507f47d8d0199bca6c9b4769048d8f5f70901e6dc74108e6433698fdb42f2"} Jul 09 18:41:47.994014 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:47.993986 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 18:41:47.994397 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:47.994375 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" event={"ID":"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985","Type":"ContainerStarted","Data":"c35a83bd5d56be23f37bceafc296e31456df65bb1ea0ec7b1caa8ed56079e486"} Jul 09 18:41:47.995922 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:47.995896 2585 generic.go:356] "Generic (PLEG): container finished" podID="6afc4ccb-8f0f-4246-9d5f-d873d013a3b7" containerID="dbd8616576d42818efc771e6dc132844e4fe5e6093898f3a61ea968882c7fdaa" exitCode=0 Jul 09 18:41:47.996030 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:47.995979 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6dnm" event={"ID":"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7","Type":"ContainerDied","Data":"dbd8616576d42818efc771e6dc132844e4fe5e6093898f3a61ea968882c7fdaa"} Jul 09 18:41:47.997517 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:47.997419 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vswpc" event={"ID":"a21694d7-e996-4434-ba98-4dbfed837bd2","Type":"ContainerStarted","Data":"06e68e7712078ea3854e8929bc376121d51529b729015bc8e34e15cc59f1f0de"} Jul 09 18:41:47.999121 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:47.999090 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mwvlh" event={"ID":"fbaa85e6-6967-47f5-b3d7-d6ee9f54876a","Type":"ContainerStarted","Data":"52669ab6318767b5dfe10f02e37dbcb53279b4d80974e6597170171cff40002d"} Jul 09 18:41:48.000623 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:48.000602 2585 generic.go:356] "Generic (PLEG): container finished" podID="53292fd96f441b9b041e7aa978696a19" containerID="f842f9f68fc88185b121b06a2823c90255b8b4dcc18cd2d8b9beda5e0808f2f1" exitCode=0 Jul 09 18:41:48.000751 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:48.000650 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" event={"ID":"53292fd96f441b9b041e7aa978696a19","Type":"ContainerDied","Data":"f842f9f68fc88185b121b06a2823c90255b8b4dcc18cd2d8b9beda5e0808f2f1"} Jul 09 18:41:48.018246 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:48.018190 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wb72f" podStartSLOduration=3.044850587 podStartE2EDuration="24.018177259s" totalImagesPullingTime="20.973326672s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:41:48.002883858 +0000 UTC m=+24.746866980" watchObservedRunningTime="2026-07-09 18:41:48.018177259 +0000 UTC m=+24.762160372" Jul 09 18:41:48.018354 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:48.018293 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vswpc" podStartSLOduration=7.568365479 podStartE2EDuration="24.018283696s" totalImagesPullingTime="16.449918217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:41:48.017831723 +0000 UTC m=+24.761814838" watchObservedRunningTime="2026-07-09 18:41:48.018283696 +0000 UTC m=+24.762266811" Jul 09 18:41:48.071305 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:48.071250 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mwvlh" podStartSLOduration=3.059171549 podStartE2EDuration="24.07123161s" totalImagesPullingTime="21.012060061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:41:48.070967285 +0000 UTC m=+24.814950400" watchObservedRunningTime="2026-07-09 18:41:48.07123161 +0000 UTC m=+24.815214746" Jul 09 18:41:48.086518 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:48.086466 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pdlv2" podStartSLOduration=7.5876621570000005 podStartE2EDuration="24.086447616s" totalImagesPullingTime="16.498785459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:41:48.085771354 +0000 UTC m=+24.829754469" watchObservedRunningTime="2026-07-09 18:41:48.086447616 +0000 UTC m=+24.830430734" Jul 09 18:41:48.392004 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:48.391983 2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 18:41:48.760608 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:48.760493 2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T18:41:48.392000265Z","UUID":"c33fc998-393a-478e-a5a3-30e7f484e52b","Handler":null,"Name":"","Endpoint":""} Jul 09 18:41:48.762147 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:48.762113 2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 18:41:48.762269 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:48.762155 2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 18:41:48.836814 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:48.836783 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:48.837022 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:48.836784 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:48.837022 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:48.836893 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:48.837022 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:48.836993 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:49.004660 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:49.004622 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" event={"ID":"53292fd96f441b9b041e7aa978696a19","Type":"ContainerStarted","Data":"8fd9897f1d0641892153761b818dc28116d47dbe1214b609eb5458fc83e701de"} Jul 09 18:41:49.008197 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:49.008168 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" event={"ID":"cd758b0d-acc8-4ca7-8ff1-672c3572c858","Type":"ContainerStarted","Data":"209465e2a4339dd5dec1557b32cdf07150bce8d5c46f28b67c124d06d0729592"} Jul 09 18:41:49.409578 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:49.409548 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pdlv2" Jul 09 18:41:49.410061 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:49.410041 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pdlv2" Jul 09 18:41:49.427394 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:49.427350 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" podStartSLOduration=24.427336891 podStartE2EDuration="24.427336891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:41:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:41:49.021398165 +0000 UTC m=+25.765381292" watchObservedRunningTime="2026-07-09 18:41:49.427336891 +0000 UTC m=+26.171320006" Jul 09 18:41:49.837325 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:49.837057 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:49.837793 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:49.837358 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:50.012226 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:50.012190 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" event={"ID":"cd758b0d-acc8-4ca7-8ff1-672c3572c858","Type":"ContainerStarted","Data":"7da2a4617bc766e692cfe3c3011bcac654c5052f36fb17bc6cf9b1f8fcd16714"} Jul 09 18:41:50.014969 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:50.014950 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 18:41:50.015349 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:50.015321 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" event={"ID":"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985","Type":"ContainerStarted","Data":"b408aba2cc1d92988e7bf6aee0d7d5b6094f6b92d3784182c200db1835cd5655"} Jul 09 18:41:50.015724 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:50.015700 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pdlv2" Jul 09 18:41:50.016196 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:50.016178 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pdlv2" Jul 09 18:41:50.030711 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:50.030672 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mjn7t" podStartSLOduration=2.828948911 podStartE2EDuration="26.030658417s" totalImagesPullingTime="23.201709506s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:41:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:41:50.030019235 +0000 UTC m=+26.774002349" watchObservedRunningTime="2026-07-09 18:41:50.030658417 +0000 UTC m=+26.774641531" Jul 09 18:41:50.836551 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:50.836522 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:50.836865 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:50.836650 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:50.836865 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:50.836709 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:50.836865 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:50.836818 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:51.837521 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:51.837006 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:51.837521 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:51.837132 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:52.024728 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:52.024530 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 18:41:52.025177 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:52.025135 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" event={"ID":"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985","Type":"ContainerStarted","Data":"792182c78c96b088d20d5c72e172ded68d59d719e6a533e944622ad152254d40"} Jul 09 18:41:52.026283 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:52.026258 2585 scope.go:117] "RemoveContainer" containerID="923982b6f11b304279772271957d95e33f9ed4b5ea5a7ee043ee2eb8192ff55d" Jul 09 18:41:52.026373 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:52.026343 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:52.026463 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:52.026430 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:52.026612 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:52.026576 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:52.046317 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:52.046222 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:52.046428 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:52.046410 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:41:52.836791 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:52.836760 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:52.836986 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:52.836766 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:52.836986 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:52.836891 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:52.837104 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:52.836980 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:53.042036 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:53.042007 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 18:41:53.042462 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:53.042403 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" event={"ID":"5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985","Type":"ContainerStarted","Data":"16a7745811910c796badd82354917140ab046e39187aa52b376f785da1afa452"} Jul 09 18:41:53.081265 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:53.081201 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" podStartSLOduration=7.77712875 podStartE2EDuration="29.0811794s" totalImagesPullingTime="21.30405065s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:41:53.079221311 +0000 UTC m=+29.823204426" watchObservedRunningTime="2026-07-09 18:41:53.0811794 +0000 UTC m=+29.825162515" Jul 09 18:41:53.837952 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:53.837917 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:53.838076 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:53.838038 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:54.195952 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:54.195913 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8mts9"] Jul 09 18:41:54.196334 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:54.196052 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:54.196334 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:54.196168 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:54.199054 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:54.199026 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qqtf8"] Jul 09 18:41:54.199153 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:54.199144 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:54.199268 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:54.199247 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:54.199827 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:54.199802 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-22w7k"] Jul 09 18:41:54.199915 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:54.199893 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:54.200006 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:54.199985 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:55.049627 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:55.049424 2585 generic.go:356] "Generic (PLEG): container finished" podID="6afc4ccb-8f0f-4246-9d5f-d873d013a3b7" containerID="2de183c6cbf879212563ca741431ab1dddd6425e072f16c608a720686d89cdc1" exitCode=0 Jul 09 18:41:55.049627 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:55.049519 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6dnm" event={"ID":"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7","Type":"ContainerDied","Data":"2de183c6cbf879212563ca741431ab1dddd6425e072f16c608a720686d89cdc1"} Jul 09 18:41:55.836792 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:55.836699 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:55.836792 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:55.836750 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:55.837255 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:55.836860 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:55.837255 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:55.836937 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:55.837255 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:55.837034 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:55.837255 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:55.837148 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:56.438086 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:56.438020 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:56.438221 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:56.438133 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:41:56.438221 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:56.438183 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret podName:fda064a0-8a88-462c-9891-47a412d9c0f4 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:28.438170885 +0000 UTC m=+65.182153977 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret") pod "global-pull-secret-syncer-8mts9" (UID: "fda064a0-8a88-462c-9891-47a412d9c0f4") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:41:56.539179 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:56.539141 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcxbs\" (UniqueName: \"kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs\") pod \"network-check-target-22w7k\" (UID: \"eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6\") " pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:56.539374 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:56.539209 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs\") pod \"network-metrics-daemon-qqtf8\" (UID: \"52054539-243c-47f9-97f9-a7ff089dea64\") " pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:56.539374 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:56.539299 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:41:56.539374 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:56.539319 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:41:56.539374 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:56.539328 2585 projected.go:196] Error preparing data for projected volume kube-api-access-rcxbs for pod openshift-network-diagnostics/network-check-target-22w7k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:41:56.539374 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:56.539327 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:41:56.539374 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:56.539376 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs podName:eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:28.539362687 +0000 UTC m=+65.283345780 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcxbs" (UniqueName: "kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs") pod "network-check-target-22w7k" (UID: "eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:41:56.539631 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:56.539392 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs podName:52054539-243c-47f9-97f9-a7ff089dea64 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:28.539385667 +0000 UTC m=+65.283368759 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs") pod "network-metrics-daemon-qqtf8" (UID: "52054539-243c-47f9-97f9-a7ff089dea64") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:41:57.057641 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:57.057610 2585 generic.go:356] "Generic (PLEG): container finished" podID="6afc4ccb-8f0f-4246-9d5f-d873d013a3b7" containerID="a55ae2c102b5307dcdf8cfdd29aa5e4751000d4bd8f41536f05c94783ef67e31" exitCode=0 Jul 09 18:41:57.058328 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:57.057671 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6dnm" event={"ID":"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7","Type":"ContainerDied","Data":"a55ae2c102b5307dcdf8cfdd29aa5e4751000d4bd8f41536f05c94783ef67e31"} Jul 09 18:41:57.837231 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:57.837195 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:57.837231 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:57.837229 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:57.837470 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:57.837235 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:57.837470 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:57.837311 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:57.837470 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:57.837392 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:41:57.837470 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:57.837446 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:58.061927 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:58.061893 2585 generic.go:356] "Generic (PLEG): container finished" podID="6afc4ccb-8f0f-4246-9d5f-d873d013a3b7" containerID="737b25966b15bc14ac92c9d863cf2ccc9a1e171051353ff0fbd73be2f5993931" exitCode=0 Jul 09 18:41:58.062280 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:58.061939 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6dnm" event={"ID":"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7","Type":"ContainerDied","Data":"737b25966b15bc14ac92c9d863cf2ccc9a1e171051353ff0fbd73be2f5993931"} Jul 09 18:41:59.837407 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:59.837370 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:41:59.837900 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:59.837370 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:41:59.837900 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:59.837483 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqtf8" podUID="52054539-243c-47f9-97f9-a7ff089dea64" Jul 09 18:41:59.837900 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:41:59.837523 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:41:59.837900 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:59.837616 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-22w7k" podUID="eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6" Jul 09 18:41:59.837900 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:41:59.837700 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mts9" podUID="fda064a0-8a88-462c-9891-47a412d9c0f4" Jul 09 18:42:00.034546 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.034520 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeReady" Jul 09 18:42:00.034738 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.034701 2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 18:42:00.081156 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.081125 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7xb7b"] Jul 09 18:42:00.084897 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.084867 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:00.085101 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.085078 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-m4mmf"] Jul 09 18:42:00.087383 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.087359 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 18:42:00.087604 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.087358 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wm22z\"" Jul 09 18:42:00.087604 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.087405 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 18:42:00.089053 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.089032 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m4mmf" Jul 09 18:42:00.091380 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.091355 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 18:42:00.091497 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.091355 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 18:42:00.091558 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.091528 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 18:42:00.091645 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.091563 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9cbfv\"" Jul 09 18:42:00.095818 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.095761 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7xb7b"] Jul 09 18:42:00.098269 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.098248 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m4mmf"] Jul 09 18:42:00.172122 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.172084 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9ae055f-9c62-4b47-b584-c1941319a5ed-tmp-dir\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:00.172284 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.172126 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f882fb17-780a-4bff-a2b1-80b092a839c3-cert\") pod \"ingress-canary-m4mmf\" (UID: \"f882fb17-780a-4bff-a2b1-80b092a839c3\") " pod="openshift-ingress-canary/ingress-canary-m4mmf" Jul 09 18:42:00.172355 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.172275 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f9ae055f-9c62-4b47-b584-c1941319a5ed-config-volume\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:00.172355 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.172307 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-845f9\" (UniqueName: \"kubernetes.io/projected/f9ae055f-9c62-4b47-b584-c1941319a5ed-kube-api-access-845f9\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:00.172442 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.172356 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:00.172442 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.172388 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bstfm\" (UniqueName: \"kubernetes.io/projected/f882fb17-780a-4bff-a2b1-80b092a839c3-kube-api-access-bstfm\") pod \"ingress-canary-m4mmf\" (UID: \"f882fb17-780a-4bff-a2b1-80b092a839c3\") " pod="openshift-ingress-canary/ingress-canary-m4mmf" Jul 09 18:42:00.272807 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.272768 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f9ae055f-9c62-4b47-b584-c1941319a5ed-config-volume\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:00.272978 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.272816 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-845f9\" (UniqueName: \"kubernetes.io/projected/f9ae055f-9c62-4b47-b584-c1941319a5ed-kube-api-access-845f9\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:00.273038 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.272984 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:00.273096 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.273022 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bstfm\" (UniqueName: \"kubernetes.io/projected/f882fb17-780a-4bff-a2b1-80b092a839c3-kube-api-access-bstfm\") pod \"ingress-canary-m4mmf\" (UID: \"f882fb17-780a-4bff-a2b1-80b092a839c3\") " pod="openshift-ingress-canary/ingress-canary-m4mmf" Jul 09 18:42:00.273165 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.273094 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9ae055f-9c62-4b47-b584-c1941319a5ed-tmp-dir\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:00.273165 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.273120 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f882fb17-780a-4bff-a2b1-80b092a839c3-cert\") pod \"ingress-canary-m4mmf\" (UID: \"f882fb17-780a-4bff-a2b1-80b092a839c3\") " pod="openshift-ingress-canary/ingress-canary-m4mmf" Jul 09 18:42:00.273335 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:00.273312 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:42:00.273389 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:00.273371 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls podName:f9ae055f-9c62-4b47-b584-c1941319a5ed nodeName:}" failed. No retries permitted until 2026-07-09 18:42:00.773356904 +0000 UTC m=+37.517339996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls") pod "dns-default-7xb7b" (UID: "f9ae055f-9c62-4b47-b584-c1941319a5ed") : secret "dns-default-metrics-tls" not found Jul 09 18:42:00.273478 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.273459 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f9ae055f-9c62-4b47-b584-c1941319a5ed-config-volume\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:00.273560 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.273531 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9ae055f-9c62-4b47-b584-c1941319a5ed-tmp-dir\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:00.278514 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.278485 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f882fb17-780a-4bff-a2b1-80b092a839c3-cert\") pod \"ingress-canary-m4mmf\" (UID: \"f882fb17-780a-4bff-a2b1-80b092a839c3\") " pod="openshift-ingress-canary/ingress-canary-m4mmf" Jul 09 18:42:00.283120 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.283086 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-845f9\" (UniqueName: \"kubernetes.io/projected/f9ae055f-9c62-4b47-b584-c1941319a5ed-kube-api-access-845f9\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:00.284198 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.284178 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bstfm\" (UniqueName: \"kubernetes.io/projected/f882fb17-780a-4bff-a2b1-80b092a839c3-kube-api-access-bstfm\") pod \"ingress-canary-m4mmf\" (UID: \"f882fb17-780a-4bff-a2b1-80b092a839c3\") " pod="openshift-ingress-canary/ingress-canary-m4mmf" Jul 09 18:42:00.403433 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.403360 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m4mmf" Jul 09 18:42:00.578352 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.578178 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m4mmf"] Jul 09 18:42:00.582009 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:00.581984 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf882fb17_780a_4bff_a2b1_80b092a839c3.slice/crio-e7a436fbd2aad774a8dbbb7a999b4ad7623648a5318e969d54a9aa6ca2ad90f9 WatchSource:0}: Error finding container e7a436fbd2aad774a8dbbb7a999b4ad7623648a5318e969d54a9aa6ca2ad90f9: Status 404 returned error can't find the container with id e7a436fbd2aad774a8dbbb7a999b4ad7623648a5318e969d54a9aa6ca2ad90f9 Jul 09 18:42:00.777821 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:00.777785 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:00.777984 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:00.777909 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:42:00.777984 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:00.777967 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls podName:f9ae055f-9c62-4b47-b584-c1941319a5ed nodeName:}" failed. No retries permitted until 2026-07-09 18:42:01.777949691 +0000 UTC m=+38.521932804 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls") pod "dns-default-7xb7b" (UID: "f9ae055f-9c62-4b47-b584-c1941319a5ed") : secret "dns-default-metrics-tls" not found Jul 09 18:42:01.069858 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:01.069771 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m4mmf" event={"ID":"f882fb17-780a-4bff-a2b1-80b092a839c3","Type":"ContainerStarted","Data":"e7a436fbd2aad774a8dbbb7a999b4ad7623648a5318e969d54a9aa6ca2ad90f9"} Jul 09 18:42:01.788432 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:01.788389 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:01.788625 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:01.788557 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:42:01.788697 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:01.788654 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls podName:f9ae055f-9c62-4b47-b584-c1941319a5ed nodeName:}" failed. No retries permitted until 2026-07-09 18:42:03.788633622 +0000 UTC m=+40.532616733 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls") pod "dns-default-7xb7b" (UID: "f9ae055f-9c62-4b47-b584-c1941319a5ed") : secret "dns-default-metrics-tls" not found Jul 09 18:42:01.836782 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:01.836738 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:42:01.837220 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:01.836760 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:42:01.837220 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:01.836923 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:42:01.840145 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:01.840120 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 18:42:01.840507 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:01.840487 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 18:42:01.841172 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:01.840733 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 18:42:01.841172 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:01.840910 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ppzx8\"" Jul 09 18:42:01.841172 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:01.841019 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 18:42:01.841172 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:01.841074 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cz2v8\"" Jul 09 18:42:03.806828 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:03.806780 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:03.807274 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:03.806955 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:42:03.807274 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:03.807035 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls podName:f9ae055f-9c62-4b47-b584-c1941319a5ed nodeName:}" failed. No retries permitted until 2026-07-09 18:42:07.807015067 +0000 UTC m=+44.550998165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls") pod "dns-default-7xb7b" (UID: "f9ae055f-9c62-4b47-b584-c1941319a5ed") : secret "dns-default-metrics-tls" not found Jul 09 18:42:05.080881 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:05.080832 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m4mmf" event={"ID":"f882fb17-780a-4bff-a2b1-80b092a839c3","Type":"ContainerStarted","Data":"d5c9a1b9af6623c75154b597bb339da0922574afb006aa0f02eec3c92b08b3fc"} Jul 09 18:42:05.083341 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:05.083316 2585 generic.go:356] "Generic (PLEG): container finished" podID="6afc4ccb-8f0f-4246-9d5f-d873d013a3b7" containerID="afce3c1f5e7c3784603c7304b97cd9e6d07a216afd7683ccef26bef28bb96f61" exitCode=0 Jul 09 18:42:05.083464 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:05.083356 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6dnm" event={"ID":"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7","Type":"ContainerDied","Data":"afce3c1f5e7c3784603c7304b97cd9e6d07a216afd7683ccef26bef28bb96f61"} Jul 09 18:42:05.098503 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:05.098458 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-m4mmf" podStartSLOduration=1.374224589 podStartE2EDuration="5.098444755s" totalImagesPullingTime="3.724220166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:42:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:42:05.097858499 +0000 UTC m=+41.841841613" watchObservedRunningTime="2026-07-09 18:42:05.098444755 +0000 UTC m=+41.842427860" Jul 09 18:42:06.088449 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:06.088418 2585 generic.go:356] "Generic (PLEG): container finished" podID="6afc4ccb-8f0f-4246-9d5f-d873d013a3b7" containerID="4768073331afb0a159b7b0793485abfaaf6fdcc6532bc743b63eda5eb1743647" exitCode=0 Jul 09 18:42:06.088810 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:06.088478 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6dnm" event={"ID":"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7","Type":"ContainerDied","Data":"4768073331afb0a159b7b0793485abfaaf6fdcc6532bc743b63eda5eb1743647"} Jul 09 18:42:07.094383 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:07.094349 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6dnm" event={"ID":"6afc4ccb-8f0f-4246-9d5f-d873d013a3b7","Type":"ContainerStarted","Data":"95a76bbc361bc33f6158bd864d562445a0e981bcf9540f429eb462d16181786e"} Jul 09 18:42:07.121731 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:07.121671 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p6dnm" podStartSLOduration=8.040079319 podStartE2EDuration="43.121649688s" totalImagesPullingTime="35.081570369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:41:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:42:07.119909794 +0000 UTC m=+43.863892909" watchObservedRunningTime="2026-07-09 18:42:07.121649688 +0000 UTC m=+43.865632806" Jul 09 18:42:07.836239 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:07.836205 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:07.836384 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:07.836320 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:42:07.836384 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:07.836380 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls podName:f9ae055f-9c62-4b47-b584-c1941319a5ed nodeName:}" failed. No retries permitted until 2026-07-09 18:42:15.836366951 +0000 UTC m=+52.580350043 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls") pod "dns-default-7xb7b" (UID: "f9ae055f-9c62-4b47-b584-c1941319a5ed") : secret "dns-default-metrics-tls" not found Jul 09 18:42:10.797945 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:10.797914 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl"] Jul 09 18:42:10.833508 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:10.833480 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:10.839652 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:10.839622 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl"] Jul 09 18:42:10.845067 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:10.845045 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 18:42:10.848681 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:10.848661 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 09 18:42:10.853038 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:10.853012 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 18:42:10.866298 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:10.866281 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 09 18:42:10.869973 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:10.869958 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-s7j5w\"" Jul 09 18:42:10.958317 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:10.958289 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/19620443-1eb7-4610-b9fb-11aa31300c89-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-bs9hl\" (UID: \"19620443-1eb7-4610-b9fb-11aa31300c89\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:10.958467 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:10.958326 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6bjz\" (UniqueName: \"kubernetes.io/projected/19620443-1eb7-4610-b9fb-11aa31300c89-kube-api-access-g6bjz\") pod \"cluster-monitoring-operator-5f7f494dd4-bs9hl\" (UID: \"19620443-1eb7-4610-b9fb-11aa31300c89\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:10.958467 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:10.958396 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bs9hl\" (UID: \"19620443-1eb7-4610-b9fb-11aa31300c89\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:11.007660 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.007630 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr"] Jul 09 18:42:11.031384 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.031360 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-qpgdh"] Jul 09 18:42:11.031541 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.031480 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" Jul 09 18:42:11.034479 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.034449 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 09 18:42:11.034606 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.034499 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 09 18:42:11.034606 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.034543 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-4n76m\"" Jul 09 18:42:11.034813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.034796 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 09 18:42:11.034895 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.034849 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 09 18:42:11.047547 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.047526 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-dd6cd8b87-5pbbf"] Jul 09 18:42:11.047547 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.047541 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" Jul 09 18:42:11.050327 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.050267 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 09 18:42:11.050327 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.050321 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tsqbj\"" Jul 09 18:42:11.050484 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.050278 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 09 18:42:11.059195 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.059172 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bs9hl\" (UID: \"19620443-1eb7-4610-b9fb-11aa31300c89\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:11.059274 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.059234 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/19620443-1eb7-4610-b9fb-11aa31300c89-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-bs9hl\" (UID: \"19620443-1eb7-4610-b9fb-11aa31300c89\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:11.059274 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.059265 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6bjz\" (UniqueName: \"kubernetes.io/projected/19620443-1eb7-4610-b9fb-11aa31300c89-kube-api-access-g6bjz\") pod \"cluster-monitoring-operator-5f7f494dd4-bs9hl\" (UID: \"19620443-1eb7-4610-b9fb-11aa31300c89\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:11.059346 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:11.059307 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 18:42:11.059391 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:11.059379 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls podName:19620443-1eb7-4610-b9fb-11aa31300c89 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:11.559362003 +0000 UTC m=+48.303345107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bs9hl" (UID: "19620443-1eb7-4610-b9fb-11aa31300c89") : secret "cluster-monitoring-operator-tls" not found Jul 09 18:42:11.059886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.059870 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/19620443-1eb7-4610-b9fb-11aa31300c89-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-bs9hl\" (UID: \"19620443-1eb7-4610-b9fb-11aa31300c89\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:11.069622 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.069604 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-qpgdh"] Jul 09 18:42:11.069705 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.069640 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr"] Jul 09 18:42:11.069705 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.069661 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dd6cd8b87-5pbbf"] Jul 09 18:42:11.069705 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.069703 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.071909 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.071889 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 18:42:11.072519 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.072499 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 18:42:11.072632 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.072536 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 18:42:11.072803 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.072786 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xwt7z\"" Jul 09 18:42:11.076442 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.076423 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6bjz\" (UniqueName: \"kubernetes.io/projected/19620443-1eb7-4610-b9fb-11aa31300c89-kube-api-access-g6bjz\") pod \"cluster-monitoring-operator-5f7f494dd4-bs9hl\" (UID: \"19620443-1eb7-4610-b9fb-11aa31300c89\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:11.081383 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.081357 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 18:42:11.160532 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.160500 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/424fc5f3-4cd6-4463-85b3-ec78268f6ede-serving-cert\") pod \"service-ca-operator-7695fc6c8c-mhttr\" (UID: \"424fc5f3-4cd6-4463-85b3-ec78268f6ede\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" Jul 09 18:42:11.160532 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.160533 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.160730 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.160573 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjg6x\" (UniqueName: \"kubernetes.io/projected/424fc5f3-4cd6-4463-85b3-ec78268f6ede-kube-api-access-wjg6x\") pod \"service-ca-operator-7695fc6c8c-mhttr\" (UID: \"424fc5f3-4cd6-4463-85b3-ec78268f6ede\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" Jul 09 18:42:11.160730 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.160627 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/027f8931-9686-4794-af63-3f3662f5aba5-trusted-ca\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.160730 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.160644 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/027f8931-9686-4794-af63-3f3662f5aba5-installation-pull-secrets\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.160730 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.160663 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/027f8931-9686-4794-af63-3f3662f5aba5-ca-trust-extracted\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.160730 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.160677 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qpgdh\" (UID: \"383d00ba-23bb-45b1-a968-71b5dda948c5\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" Jul 09 18:42:11.160730 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.160703 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/027f8931-9686-4794-af63-3f3662f5aba5-registry-certificates\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.160907 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.160745 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/383d00ba-23bb-45b1-a968-71b5dda948c5-nginx-conf\") pod \"networking-console-plugin-6cf856758c-qpgdh\" (UID: \"383d00ba-23bb-45b1-a968-71b5dda948c5\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" Jul 09 18:42:11.160907 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.160816 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/424fc5f3-4cd6-4463-85b3-ec78268f6ede-config\") pod \"service-ca-operator-7695fc6c8c-mhttr\" (UID: \"424fc5f3-4cd6-4463-85b3-ec78268f6ede\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" Jul 09 18:42:11.160907 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.160835 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/027f8931-9686-4794-af63-3f3662f5aba5-image-registry-private-configuration\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.160907 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.160859 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-bound-sa-token\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.160907 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.160876 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njnq4\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-kube-api-access-njnq4\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.262008 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.261979 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/027f8931-9686-4794-af63-3f3662f5aba5-trusted-ca\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.262008 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.262011 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/027f8931-9686-4794-af63-3f3662f5aba5-installation-pull-secrets\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.262202 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.262031 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/027f8931-9686-4794-af63-3f3662f5aba5-ca-trust-extracted\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.262202 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.262047 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qpgdh\" (UID: \"383d00ba-23bb-45b1-a968-71b5dda948c5\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" Jul 09 18:42:11.262202 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.262070 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/027f8931-9686-4794-af63-3f3662f5aba5-registry-certificates\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.262202 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.262093 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/383d00ba-23bb-45b1-a968-71b5dda948c5-nginx-conf\") pod \"networking-console-plugin-6cf856758c-qpgdh\" (UID: \"383d00ba-23bb-45b1-a968-71b5dda948c5\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" Jul 09 18:42:11.262202 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.262129 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/424fc5f3-4cd6-4463-85b3-ec78268f6ede-config\") pod \"service-ca-operator-7695fc6c8c-mhttr\" (UID: \"424fc5f3-4cd6-4463-85b3-ec78268f6ede\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" Jul 09 18:42:11.262202 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.262153 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/027f8931-9686-4794-af63-3f3662f5aba5-image-registry-private-configuration\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.262202 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.262174 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-bound-sa-token\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.262202 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:11.262177 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 18:42:11.262202 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.262197 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njnq4\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-kube-api-access-njnq4\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.262576 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.262294 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/424fc5f3-4cd6-4463-85b3-ec78268f6ede-serving-cert\") pod \"service-ca-operator-7695fc6c8c-mhttr\" (UID: \"424fc5f3-4cd6-4463-85b3-ec78268f6ede\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" Jul 09 18:42:11.262576 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:11.262326 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert podName:383d00ba-23bb-45b1-a968-71b5dda948c5 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:11.762304661 +0000 UTC m=+48.506287777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-qpgdh" (UID: "383d00ba-23bb-45b1-a968-71b5dda948c5") : secret "networking-console-plugin-cert" not found Jul 09 18:42:11.262576 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.262348 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.262576 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.262401 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjg6x\" (UniqueName: \"kubernetes.io/projected/424fc5f3-4cd6-4463-85b3-ec78268f6ede-kube-api-access-wjg6x\") pod \"service-ca-operator-7695fc6c8c-mhttr\" (UID: \"424fc5f3-4cd6-4463-85b3-ec78268f6ede\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" Jul 09 18:42:11.262576 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.262509 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/027f8931-9686-4794-af63-3f3662f5aba5-ca-trust-extracted\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.262870 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:11.262810 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 18:42:11.262870 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:11.262827 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dd6cd8b87-5pbbf: secret "image-registry-tls" not found Jul 09 18:42:11.262870 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:11.262871 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls podName:027f8931-9686-4794-af63-3f3662f5aba5 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:11.76285514 +0000 UTC m=+48.506838232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls") pod "image-registry-dd6cd8b87-5pbbf" (UID: "027f8931-9686-4794-af63-3f3662f5aba5") : secret "image-registry-tls" not found Jul 09 18:42:11.263033 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.262871 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/027f8931-9686-4794-af63-3f3662f5aba5-registry-certificates\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.263190 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.263158 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/383d00ba-23bb-45b1-a968-71b5dda948c5-nginx-conf\") pod \"networking-console-plugin-6cf856758c-qpgdh\" (UID: \"383d00ba-23bb-45b1-a968-71b5dda948c5\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" Jul 09 18:42:11.263984 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.263959 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/027f8931-9686-4794-af63-3f3662f5aba5-trusted-ca\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.264767 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.264744 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/424fc5f3-4cd6-4463-85b3-ec78268f6ede-serving-cert\") pod \"service-ca-operator-7695fc6c8c-mhttr\" (UID: \"424fc5f3-4cd6-4463-85b3-ec78268f6ede\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" Jul 09 18:42:11.264863 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.264749 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/027f8931-9686-4794-af63-3f3662f5aba5-installation-pull-secrets\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.265032 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.265005 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/027f8931-9686-4794-af63-3f3662f5aba5-image-registry-private-configuration\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.272269 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.272248 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-bound-sa-token\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.272646 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.272626 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjg6x\" (UniqueName: \"kubernetes.io/projected/424fc5f3-4cd6-4463-85b3-ec78268f6ede-kube-api-access-wjg6x\") pod \"service-ca-operator-7695fc6c8c-mhttr\" (UID: \"424fc5f3-4cd6-4463-85b3-ec78268f6ede\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" Jul 09 18:42:11.272801 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.272785 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njnq4\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-kube-api-access-njnq4\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.273163 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.273143 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/424fc5f3-4cd6-4463-85b3-ec78268f6ede-config\") pod \"service-ca-operator-7695fc6c8c-mhttr\" (UID: \"424fc5f3-4cd6-4463-85b3-ec78268f6ede\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" Jul 09 18:42:11.341715 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.341638 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" Jul 09 18:42:11.483387 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.483356 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr"] Jul 09 18:42:11.486047 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:11.486025 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod424fc5f3_4cd6_4463_85b3_ec78268f6ede.slice/crio-b5411804bd21ae6ec7d79f9d9aa53b18af16dbd262b7ebfff717c7e39f86cfff WatchSource:0}: Error finding container b5411804bd21ae6ec7d79f9d9aa53b18af16dbd262b7ebfff717c7e39f86cfff: Status 404 returned error can't find the container with id b5411804bd21ae6ec7d79f9d9aa53b18af16dbd262b7ebfff717c7e39f86cfff Jul 09 18:42:11.565306 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.565277 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bs9hl\" (UID: \"19620443-1eb7-4610-b9fb-11aa31300c89\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:11.565476 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:11.565441 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 18:42:11.565523 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:11.565502 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls podName:19620443-1eb7-4610-b9fb-11aa31300c89 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:12.565487634 +0000 UTC m=+49.309470728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bs9hl" (UID: "19620443-1eb7-4610-b9fb-11aa31300c89") : secret "cluster-monitoring-operator-tls" not found Jul 09 18:42:11.767435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.767396 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:11.767655 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:11.767454 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qpgdh\" (UID: \"383d00ba-23bb-45b1-a968-71b5dda948c5\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" Jul 09 18:42:11.767655 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:11.767550 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 18:42:11.767655 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:11.767562 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 18:42:11.767655 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:11.767609 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dd6cd8b87-5pbbf: secret "image-registry-tls" not found Jul 09 18:42:11.767655 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:11.767617 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert podName:383d00ba-23bb-45b1-a968-71b5dda948c5 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:12.767603072 +0000 UTC m=+49.511586165 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-qpgdh" (UID: "383d00ba-23bb-45b1-a968-71b5dda948c5") : secret "networking-console-plugin-cert" not found Jul 09 18:42:11.767843 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:11.767663 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls podName:027f8931-9686-4794-af63-3f3662f5aba5 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:12.767645693 +0000 UTC m=+49.511628791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls") pod "image-registry-dd6cd8b87-5pbbf" (UID: "027f8931-9686-4794-af63-3f3662f5aba5") : secret "image-registry-tls" not found Jul 09 18:42:12.106743 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:12.106663 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" event={"ID":"424fc5f3-4cd6-4463-85b3-ec78268f6ede","Type":"ContainerStarted","Data":"b5411804bd21ae6ec7d79f9d9aa53b18af16dbd262b7ebfff717c7e39f86cfff"} Jul 09 18:42:12.573730 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:12.573685 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bs9hl\" (UID: \"19620443-1eb7-4610-b9fb-11aa31300c89\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:12.573919 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:12.573831 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 18:42:12.573919 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:12.573902 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls podName:19620443-1eb7-4610-b9fb-11aa31300c89 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:14.573887416 +0000 UTC m=+51.317870508 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bs9hl" (UID: "19620443-1eb7-4610-b9fb-11aa31300c89") : secret "cluster-monitoring-operator-tls" not found Jul 09 18:42:12.775436 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:12.775392 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:12.775617 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:12.775482 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qpgdh\" (UID: \"383d00ba-23bb-45b1-a968-71b5dda948c5\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" Jul 09 18:42:12.775617 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:12.775496 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 18:42:12.775617 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:12.775517 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dd6cd8b87-5pbbf: secret "image-registry-tls" not found Jul 09 18:42:12.775617 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:12.775604 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls podName:027f8931-9686-4794-af63-3f3662f5aba5 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:14.775563644 +0000 UTC m=+51.519546757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls") pod "image-registry-dd6cd8b87-5pbbf" (UID: "027f8931-9686-4794-af63-3f3662f5aba5") : secret "image-registry-tls" not found Jul 09 18:42:12.775617 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:12.775608 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 18:42:12.775800 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:12.775644 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert podName:383d00ba-23bb-45b1-a968-71b5dda948c5 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:14.775633638 +0000 UTC m=+51.519616730 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-qpgdh" (UID: "383d00ba-23bb-45b1-a968-71b5dda948c5") : secret "networking-console-plugin-cert" not found Jul 09 18:42:14.589829 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:14.589791 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bs9hl\" (UID: \"19620443-1eb7-4610-b9fb-11aa31300c89\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:14.590274 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:14.589936 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 18:42:14.590274 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:14.590002 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls podName:19620443-1eb7-4610-b9fb-11aa31300c89 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:18.589987061 +0000 UTC m=+55.333970153 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bs9hl" (UID: "19620443-1eb7-4610-b9fb-11aa31300c89") : secret "cluster-monitoring-operator-tls" not found Jul 09 18:42:14.791440 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:14.791406 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:14.791615 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:14.791458 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qpgdh\" (UID: \"383d00ba-23bb-45b1-a968-71b5dda948c5\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" Jul 09 18:42:14.791615 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:14.791549 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 18:42:14.791615 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:14.791554 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 18:42:14.791615 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:14.791571 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dd6cd8b87-5pbbf: secret "image-registry-tls" not found Jul 09 18:42:14.791615 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:14.791618 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert podName:383d00ba-23bb-45b1-a968-71b5dda948c5 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:18.791601598 +0000 UTC m=+55.535584717 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-qpgdh" (UID: "383d00ba-23bb-45b1-a968-71b5dda948c5") : secret "networking-console-plugin-cert" not found Jul 09 18:42:14.791788 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:14.791634 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls podName:027f8931-9686-4794-af63-3f3662f5aba5 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:18.791626056 +0000 UTC m=+55.535609147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls") pod "image-registry-dd6cd8b87-5pbbf" (UID: "027f8931-9686-4794-af63-3f3662f5aba5") : secret "image-registry-tls" not found Jul 09 18:42:15.115885 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:15.115851 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" event={"ID":"424fc5f3-4cd6-4463-85b3-ec78268f6ede","Type":"ContainerStarted","Data":"f1bbb25dd3d3fac6504853037197458196bc0f9be80f47628f99dc7a4de36f85"} Jul 09 18:42:15.133341 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:15.133294 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" podStartSLOduration=1.6425261789999999 podStartE2EDuration="5.133260051s" totalImagesPullingTime="3.490733872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:42:15.13247994 +0000 UTC m=+51.876463067" watchObservedRunningTime="2026-07-09 18:42:15.133260051 +0000 UTC m=+51.877243164" Jul 09 18:42:15.900496 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:15.900463 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:15.900946 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:15.900634 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:42:15.900946 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:15.900697 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls podName:f9ae055f-9c62-4b47-b584-c1941319a5ed nodeName:}" failed. No retries permitted until 2026-07-09 18:42:31.900681331 +0000 UTC m=+68.644664423 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls") pod "dns-default-7xb7b" (UID: "f9ae055f-9c62-4b47-b584-c1941319a5ed") : secret "dns-default-metrics-tls" not found Jul 09 18:42:15.963679 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:15.963646 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r8dks"] Jul 09 18:42:15.985670 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:15.985637 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:15.987604 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:15.987554 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r8dks"] Jul 09 18:42:15.988702 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:15.988675 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-z2rdm\"" Jul 09 18:42:15.988846 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:15.988721 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 18:42:15.989681 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:15.989654 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 18:42:15.989681 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:15.989673 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 18:42:15.989829 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:15.989673 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 18:42:16.101853 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:16.101824 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:16.102034 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:16.101885 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:16.102034 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:16.101940 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-data-volume\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:16.102034 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:16.101964 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvknn\" (UniqueName: \"kubernetes.io/projected/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-api-access-wvknn\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:16.102189 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:16.102054 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-crio-socket\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:16.202952 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:16.202910 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-crio-socket\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:16.203145 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:16.202981 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:16.203145 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:16.203040 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:16.203145 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:16.203090 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-data-volume\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:16.203145 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:16.203105 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 18:42:16.203145 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:16.203125 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvknn\" (UniqueName: \"kubernetes.io/projected/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-api-access-wvknn\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:16.203145 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:16.203136 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-crio-socket\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:16.203453 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:16.203188 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm podName:e7275e4d-1d71-4b91-9ae2-a78ef40557e7 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:16.70317262 +0000 UTC m=+53.447155731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r8dks" (UID: "e7275e4d-1d71-4b91-9ae2-a78ef40557e7") : configmap "kube-rbac-proxy" not found Jul 09 18:42:16.203453 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:16.203220 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 18:42:16.203453 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:16.203283 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls podName:e7275e4d-1d71-4b91-9ae2-a78ef40557e7 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:16.703266844 +0000 UTC m=+53.447249944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r8dks" (UID: "e7275e4d-1d71-4b91-9ae2-a78ef40557e7") : secret "insights-runtime-extractor-tls" not found Jul 09 18:42:16.203453 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:16.203420 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-data-volume\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:16.214517 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:16.214484 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvknn\" (UniqueName: \"kubernetes.io/projected/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-api-access-wvknn\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:16.707652 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:16.707619 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:16.707800 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:16.707685 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:16.707800 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:16.707740 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 18:42:16.707800 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:16.707776 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 18:42:16.707917 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:16.707806 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm podName:e7275e4d-1d71-4b91-9ae2-a78ef40557e7 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:17.707790801 +0000 UTC m=+54.451773893 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r8dks" (UID: "e7275e4d-1d71-4b91-9ae2-a78ef40557e7") : configmap "kube-rbac-proxy" not found Jul 09 18:42:16.707917 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:16.707825 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls podName:e7275e4d-1d71-4b91-9ae2-a78ef40557e7 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:17.707813607 +0000 UTC m=+54.451796701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r8dks" (UID: "e7275e4d-1d71-4b91-9ae2-a78ef40557e7") : secret "insights-runtime-extractor-tls" not found Jul 09 18:42:17.715874 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:17.715837 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:17.716249 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:17.715895 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:17.716249 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:17.715968 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 18:42:17.716249 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:17.715996 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 18:42:17.716249 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:17.716040 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls podName:e7275e4d-1d71-4b91-9ae2-a78ef40557e7 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:19.716025855 +0000 UTC m=+56.460008947 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r8dks" (UID: "e7275e4d-1d71-4b91-9ae2-a78ef40557e7") : secret "insights-runtime-extractor-tls" not found Jul 09 18:42:17.716249 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:17.716053 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm podName:e7275e4d-1d71-4b91-9ae2-a78ef40557e7 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:19.716046719 +0000 UTC m=+56.460029810 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r8dks" (UID: "e7275e4d-1d71-4b91-9ae2-a78ef40557e7") : configmap "kube-rbac-proxy" not found Jul 09 18:42:18.053792 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.053711 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-l5rj8"] Jul 09 18:42:18.069626 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.069574 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-l5rj8"] Jul 09 18:42:18.069791 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.069691 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-l5rj8" Jul 09 18:42:18.073319 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.073300 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 09 18:42:18.073453 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.073344 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 09 18:42:18.073935 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.073920 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 09 18:42:18.074342 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.074327 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9hf2v\"" Jul 09 18:42:18.074342 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.074337 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 09 18:42:18.120129 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.120099 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176-signing-key\") pod \"service-ca-79f68b9f4f-l5rj8\" (UID: \"7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l5rj8" Jul 09 18:42:18.120275 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.120133 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t55f5\" (UniqueName: \"kubernetes.io/projected/7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176-kube-api-access-t55f5\") pod \"service-ca-79f68b9f4f-l5rj8\" (UID: \"7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l5rj8" Jul 09 18:42:18.120275 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.120153 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176-signing-cabundle\") pod \"service-ca-79f68b9f4f-l5rj8\" (UID: \"7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l5rj8" Jul 09 18:42:18.128954 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.128931 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wb72f_397a8531-67a9-4a3f-ad6b-f450a14856a3/dns-node-resolver/0.log" Jul 09 18:42:18.221141 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.221109 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176-signing-key\") pod \"service-ca-79f68b9f4f-l5rj8\" (UID: \"7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l5rj8" Jul 09 18:42:18.221296 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.221147 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t55f5\" (UniqueName: \"kubernetes.io/projected/7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176-kube-api-access-t55f5\") pod \"service-ca-79f68b9f4f-l5rj8\" (UID: \"7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l5rj8" Jul 09 18:42:18.221296 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.221166 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176-signing-cabundle\") pod \"service-ca-79f68b9f4f-l5rj8\" (UID: \"7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l5rj8" Jul 09 18:42:18.221816 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.221795 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176-signing-cabundle\") pod \"service-ca-79f68b9f4f-l5rj8\" (UID: \"7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l5rj8" Jul 09 18:42:18.223651 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.223630 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176-signing-key\") pod \"service-ca-79f68b9f4f-l5rj8\" (UID: \"7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l5rj8" Jul 09 18:42:18.231977 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.231952 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t55f5\" (UniqueName: \"kubernetes.io/projected/7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176-kube-api-access-t55f5\") pod \"service-ca-79f68b9f4f-l5rj8\" (UID: \"7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l5rj8" Jul 09 18:42:18.378714 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.378644 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-l5rj8" Jul 09 18:42:18.527688 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.527659 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-l5rj8"] Jul 09 18:42:18.530781 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:18.530751 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bd228b5_6a69_4aaf_a0ef_3a69ab5d2176.slice/crio-3eda5b83c5a066fc6e4ef366762dc9e9eb3d0f32512fc277a80dc5d883bfe8a1 WatchSource:0}: Error finding container 3eda5b83c5a066fc6e4ef366762dc9e9eb3d0f32512fc277a80dc5d883bfe8a1: Status 404 returned error can't find the container with id 3eda5b83c5a066fc6e4ef366762dc9e9eb3d0f32512fc277a80dc5d883bfe8a1 Jul 09 18:42:18.624501 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.624474 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bs9hl\" (UID: \"19620443-1eb7-4610-b9fb-11aa31300c89\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:18.624666 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:18.624648 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 18:42:18.624730 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:18.624720 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls podName:19620443-1eb7-4610-b9fb-11aa31300c89 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:26.624705132 +0000 UTC m=+63.368688227 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bs9hl" (UID: "19620443-1eb7-4610-b9fb-11aa31300c89") : secret "cluster-monitoring-operator-tls" not found Jul 09 18:42:18.826460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.826422 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:18.826878 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:18.826495 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qpgdh\" (UID: \"383d00ba-23bb-45b1-a968-71b5dda948c5\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" Jul 09 18:42:18.826878 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:18.826566 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 18:42:18.826878 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:18.826600 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dd6cd8b87-5pbbf: secret "image-registry-tls" not found Jul 09 18:42:18.826878 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:18.826627 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 18:42:18.826878 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:18.826663 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls podName:027f8931-9686-4794-af63-3f3662f5aba5 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:26.826649067 +0000 UTC m=+63.570632161 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls") pod "image-registry-dd6cd8b87-5pbbf" (UID: "027f8931-9686-4794-af63-3f3662f5aba5") : secret "image-registry-tls" not found Jul 09 18:42:18.826878 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:18.826677 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert podName:383d00ba-23bb-45b1-a968-71b5dda948c5 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:26.826669899 +0000 UTC m=+63.570652991 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-qpgdh" (UID: "383d00ba-23bb-45b1-a968-71b5dda948c5") : secret "networking-console-plugin-cert" not found Jul 09 18:42:19.126065 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:19.126034 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-l5rj8" event={"ID":"7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176","Type":"ContainerStarted","Data":"e7b92e75648ddf52afdc2761608815263fcdd44000ffc92a44cb96865684798a"} Jul 09 18:42:19.126238 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:19.126076 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-l5rj8" event={"ID":"7bd228b5-6a69-4aaf-a0ef-3a69ab5d2176","Type":"ContainerStarted","Data":"3eda5b83c5a066fc6e4ef366762dc9e9eb3d0f32512fc277a80dc5d883bfe8a1"} Jul 09 18:42:19.128212 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:19.128192 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vswpc_a21694d7-e996-4434-ba98-4dbfed837bd2/node-ca/0.log" Jul 09 18:42:19.144282 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:19.144237 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-l5rj8" podStartSLOduration=1.144224625 podStartE2EDuration="1.144224625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:42:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:42:19.143428208 +0000 UTC m=+55.887411332" watchObservedRunningTime="2026-07-09 18:42:19.144224625 +0000 UTC m=+55.888207739" Jul 09 18:42:19.733652 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:19.733616 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:19.733893 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:19.733712 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:19.733893 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:19.733730 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 18:42:19.733893 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:19.733806 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm podName:e7275e4d-1d71-4b91-9ae2-a78ef40557e7 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:23.733784126 +0000 UTC m=+60.477767225 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r8dks" (UID: "e7275e4d-1d71-4b91-9ae2-a78ef40557e7") : configmap "kube-rbac-proxy" not found Jul 09 18:42:19.733893 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:19.733853 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 18:42:19.734110 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:19.733907 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls podName:e7275e4d-1d71-4b91-9ae2-a78ef40557e7 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:23.733894782 +0000 UTC m=+60.477877874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r8dks" (UID: "e7275e4d-1d71-4b91-9ae2-a78ef40557e7") : secret "insights-runtime-extractor-tls" not found Jul 09 18:42:19.929914 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:19.929875 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m4mmf_f882fb17-780a-4bff-a2b1-80b092a839c3/serve-healthcheck-canary/0.log" Jul 09 18:42:23.766990 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:23.766963 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:23.767344 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:23.767021 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:23.767344 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:23.767079 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 18:42:23.767344 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:23.767119 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 18:42:23.767344 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:23.767145 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm podName:e7275e4d-1d71-4b91-9ae2-a78ef40557e7 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:31.76712994 +0000 UTC m=+68.511113035 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r8dks" (UID: "e7275e4d-1d71-4b91-9ae2-a78ef40557e7") : configmap "kube-rbac-proxy" not found Jul 09 18:42:23.767344 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:23.767163 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls podName:e7275e4d-1d71-4b91-9ae2-a78ef40557e7 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:31.767152436 +0000 UTC m=+68.511135528 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r8dks" (UID: "e7275e4d-1d71-4b91-9ae2-a78ef40557e7") : secret "insights-runtime-extractor-tls" not found Jul 09 18:42:24.056608 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:24.056523 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzm9x" Jul 09 18:42:26.689015 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:26.688970 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bs9hl\" (UID: \"19620443-1eb7-4610-b9fb-11aa31300c89\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:26.691497 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:26.691477 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19620443-1eb7-4610-b9fb-11aa31300c89-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bs9hl\" (UID: \"19620443-1eb7-4610-b9fb-11aa31300c89\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:26.744772 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:26.744745 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-s7j5w\"" Jul 09 18:42:26.752926 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:26.752897 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" Jul 09 18:42:26.891273 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:26.891241 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:26.891424 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:26.891339 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qpgdh\" (UID: \"383d00ba-23bb-45b1-a968-71b5dda948c5\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" Jul 09 18:42:26.891512 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:26.891492 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 18:42:26.891578 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:26.891568 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert podName:383d00ba-23bb-45b1-a968-71b5dda948c5 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:42.891552508 +0000 UTC m=+79.635535618 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-qpgdh" (UID: "383d00ba-23bb-45b1-a968-71b5dda948c5") : secret "networking-console-plugin-cert" not found Jul 09 18:42:26.893841 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:26.893820 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls\") pod \"image-registry-dd6cd8b87-5pbbf\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:26.906702 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:26.906679 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl"] Jul 09 18:42:26.909892 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:26.909869 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19620443_1eb7_4610_b9fb_11aa31300c89.slice/crio-fa5673642ba729b61e3acb160fb50f22521a314397818c7dec7af8d76ff9a73c WatchSource:0}: Error finding container fa5673642ba729b61e3acb160fb50f22521a314397818c7dec7af8d76ff9a73c: Status 404 returned error can't find the container with id fa5673642ba729b61e3acb160fb50f22521a314397818c7dec7af8d76ff9a73c Jul 09 18:42:26.985922 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:26.985855 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xwt7z\"" Jul 09 18:42:26.993953 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:26.993934 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:27.127976 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:27.127948 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dd6cd8b87-5pbbf"] Jul 09 18:42:27.131009 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:27.130984 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod027f8931_9686_4794_af63_3f3662f5aba5.slice/crio-dc2f643c8ef45d175211e2c120a3276f37664a11a076ff9844a497a1c51ef5b5 WatchSource:0}: Error finding container dc2f643c8ef45d175211e2c120a3276f37664a11a076ff9844a497a1c51ef5b5: Status 404 returned error can't find the container with id dc2f643c8ef45d175211e2c120a3276f37664a11a076ff9844a497a1c51ef5b5 Jul 09 18:42:27.149119 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:27.149094 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" event={"ID":"027f8931-9686-4794-af63-3f3662f5aba5","Type":"ContainerStarted","Data":"dc2f643c8ef45d175211e2c120a3276f37664a11a076ff9844a497a1c51ef5b5"} Jul 09 18:42:27.150160 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:27.150138 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" event={"ID":"19620443-1eb7-4610-b9fb-11aa31300c89","Type":"ContainerStarted","Data":"fa5673642ba729b61e3acb160fb50f22521a314397818c7dec7af8d76ff9a73c"} Jul 09 18:42:28.155507 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.155471 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" event={"ID":"027f8931-9686-4794-af63-3f3662f5aba5","Type":"ContainerStarted","Data":"5636c328839b80f349d6facb44adbc1e7c355da88956f9384c1929e122cce502"} Jul 09 18:42:28.155998 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.155621 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:28.507287 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.507142 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:42:28.510047 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.510020 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 18:42:28.520890 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.520865 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda064a0-8a88-462c-9891-47a412d9c0f4-original-pull-secret\") pod \"global-pull-secret-syncer-8mts9\" (UID: \"fda064a0-8a88-462c-9891-47a412d9c0f4\") " pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:42:28.558066 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.558033 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mts9" Jul 09 18:42:28.608226 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.608198 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcxbs\" (UniqueName: \"kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs\") pod \"network-check-target-22w7k\" (UID: \"eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6\") " pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:42:28.608391 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.608244 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs\") pod \"network-metrics-daemon-qqtf8\" (UID: \"52054539-243c-47f9-97f9-a7ff089dea64\") " pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:42:28.610932 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.610905 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 18:42:28.611089 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.610908 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 18:42:28.621882 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.621859 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 18:42:28.622096 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.622072 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52054539-243c-47f9-97f9-a7ff089dea64-metrics-certs\") pod \"network-metrics-daemon-qqtf8\" (UID: \"52054539-243c-47f9-97f9-a7ff089dea64\") " pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:42:28.638266 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.633262 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcxbs\" (UniqueName: \"kubernetes.io/projected/eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6-kube-api-access-rcxbs\") pod \"network-check-target-22w7k\" (UID: \"eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6\") " pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:42:28.853332 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.853303 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cz2v8\"" Jul 09 18:42:28.861659 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.861631 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:42:28.866227 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.866208 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ppzx8\"" Jul 09 18:42:28.874499 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.874469 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqtf8" Jul 09 18:42:28.973565 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.973506 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" podStartSLOduration=18.97348159 podStartE2EDuration="18.97348159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:42:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:42:28.178743652 +0000 UTC m=+64.922726768" watchObservedRunningTime="2026-07-09 18:42:28.97348159 +0000 UTC m=+65.717464707" Jul 09 18:42:28.974386 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:28.974323 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8mts9"] Jul 09 18:42:28.978395 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:28.978364 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfda064a0_8a88_462c_9891_47a412d9c0f4.slice/crio-ed1724b532e19b106e5658ffd2256fa1a5731d28baf884ac055c97b0429f07bf WatchSource:0}: Error finding container ed1724b532e19b106e5658ffd2256fa1a5731d28baf884ac055c97b0429f07bf: Status 404 returned error can't find the container with id ed1724b532e19b106e5658ffd2256fa1a5731d28baf884ac055c97b0429f07bf Jul 09 18:42:29.022339 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:29.022303 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-22w7k"] Jul 09 18:42:29.025345 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:29.025319 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee9f1bb_0aa9_461e_86f6_2653ecb2a0f6.slice/crio-c308bf260892d208b484325765fd1267fd2a4da9620f3c7c8dad996641aeab10 WatchSource:0}: Error finding container c308bf260892d208b484325765fd1267fd2a4da9620f3c7c8dad996641aeab10: Status 404 returned error can't find the container with id c308bf260892d208b484325765fd1267fd2a4da9620f3c7c8dad996641aeab10 Jul 09 18:42:29.037470 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:29.037445 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qqtf8"] Jul 09 18:42:29.040114 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:29.040092 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52054539_243c_47f9_97f9_a7ff089dea64.slice/crio-7928b918cd4c0038ffbc4ed80f74568526bc42b776e18650bc7b6f1cc6f57bb8 WatchSource:0}: Error finding container 7928b918cd4c0038ffbc4ed80f74568526bc42b776e18650bc7b6f1cc6f57bb8: Status 404 returned error can't find the container with id 7928b918cd4c0038ffbc4ed80f74568526bc42b776e18650bc7b6f1cc6f57bb8 Jul 09 18:42:29.165233 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:29.165127 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" event={"ID":"19620443-1eb7-4610-b9fb-11aa31300c89","Type":"ContainerStarted","Data":"59236028f9b2e5cc7db899dbc793f3e8fc5a8a18c2c389c9685ce7f082e6af02"} Jul 09 18:42:29.166346 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:29.166318 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8mts9" event={"ID":"fda064a0-8a88-462c-9891-47a412d9c0f4","Type":"ContainerStarted","Data":"ed1724b532e19b106e5658ffd2256fa1a5731d28baf884ac055c97b0429f07bf"} Jul 09 18:42:29.168018 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:29.167485 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-22w7k" event={"ID":"eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6","Type":"ContainerStarted","Data":"c308bf260892d208b484325765fd1267fd2a4da9620f3c7c8dad996641aeab10"} Jul 09 18:42:29.169851 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:29.169813 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qqtf8" event={"ID":"52054539-243c-47f9-97f9-a7ff089dea64","Type":"ContainerStarted","Data":"7928b918cd4c0038ffbc4ed80f74568526bc42b776e18650bc7b6f1cc6f57bb8"} Jul 09 18:42:29.186837 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:29.186777 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bs9hl" podStartSLOduration=17.216693396 podStartE2EDuration="19.186759343s" totalImagesPullingTime="1.970065947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:42:29.186007507 +0000 UTC m=+65.929990622" watchObservedRunningTime="2026-07-09 18:42:29.186759343 +0000 UTC m=+65.930742458" Jul 09 18:42:31.181018 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:31.180979 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qqtf8" event={"ID":"52054539-243c-47f9-97f9-a7ff089dea64","Type":"ContainerStarted","Data":"d03f2413eaa66b31b18e65aef777954664791b0133330fbcb5012c364e12f45d"} Jul 09 18:42:31.181432 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:31.181036 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qqtf8" event={"ID":"52054539-243c-47f9-97f9-a7ff089dea64","Type":"ContainerStarted","Data":"85b1c04b1a401f330f209c36e59ad9221eeebbc281db8620cecbfa549eee1ba5"} Jul 09 18:42:31.198819 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:31.198276 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qqtf8" podStartSLOduration=65.904127061 podStartE2EDuration="1m7.198260746s" totalImagesPullingTime="1.294133685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:42:31.197611242 +0000 UTC m=+67.941594357" watchObservedRunningTime="2026-07-09 18:42:31.198260746 +0000 UTC m=+67.942243837" Jul 09 18:42:31.838887 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:31.838848 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:31.839066 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:31.838950 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:31.839576 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:31.839547 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:31.842275 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:31.842249 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7275e4d-1d71-4b91-9ae2-a78ef40557e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8dks\" (UID: \"e7275e4d-1d71-4b91-9ae2-a78ef40557e7\") " pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:31.897579 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:31.897542 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-z2rdm\"" Jul 09 18:42:31.905678 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:31.905653 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r8dks" Jul 09 18:42:31.940127 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:31.940097 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:31.942771 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:31.942745 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9ae055f-9c62-4b47-b584-c1941319a5ed-metrics-tls\") pod \"dns-default-7xb7b\" (UID: \"f9ae055f-9c62-4b47-b584-c1941319a5ed\") " pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:32.199799 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:32.199766 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wm22z\"" Jul 09 18:42:32.207916 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:32.207889 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:34.775399 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:34.775373 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7xb7b"] Jul 09 18:42:34.777974 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:34.777952 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9ae055f_9c62_4b47_b584_c1941319a5ed.slice/crio-ef2586f524a7881f6746458e4f77dd31d31b8593d34ec5d0e9c5ba15cf06dae7 WatchSource:0}: Error finding container ef2586f524a7881f6746458e4f77dd31d31b8593d34ec5d0e9c5ba15cf06dae7: Status 404 returned error can't find the container with id ef2586f524a7881f6746458e4f77dd31d31b8593d34ec5d0e9c5ba15cf06dae7 Jul 09 18:42:34.780928 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:34.780905 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r8dks"] Jul 09 18:42:34.783165 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:34.783138 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7275e4d_1d71_4b91_9ae2_a78ef40557e7.slice/crio-6601404b1c3297be9d1d563c12b22540eb92d9c172f3c947847facff71a0cf85 WatchSource:0}: Error finding container 6601404b1c3297be9d1d563c12b22540eb92d9c172f3c947847facff71a0cf85: Status 404 returned error can't find the container with id 6601404b1c3297be9d1d563c12b22540eb92d9c172f3c947847facff71a0cf85 Jul 09 18:42:35.200064 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:35.200026 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-22w7k" event={"ID":"eee9f1bb-0aa9-461e-86f6-2653ecb2a0f6","Type":"ContainerStarted","Data":"88f596c35c54009e1707227681db0e5efde55edb64c0028319cfa66eb7c31f4d"} Jul 09 18:42:35.200390 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:35.200118 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:42:35.201418 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:35.201387 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xb7b" event={"ID":"f9ae055f-9c62-4b47-b584-c1941319a5ed","Type":"ContainerStarted","Data":"ef2586f524a7881f6746458e4f77dd31d31b8593d34ec5d0e9c5ba15cf06dae7"} Jul 09 18:42:35.202966 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:35.202929 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8mts9" event={"ID":"fda064a0-8a88-462c-9891-47a412d9c0f4","Type":"ContainerStarted","Data":"fb9d3d6f1c2e36e39df0cf59733ad299ff854c00f6cf2a5b4554ed9004b57563"} Jul 09 18:42:35.204142 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:35.204118 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r8dks" event={"ID":"e7275e4d-1d71-4b91-9ae2-a78ef40557e7","Type":"ContainerStarted","Data":"6601404b1c3297be9d1d563c12b22540eb92d9c172f3c947847facff71a0cf85"} Jul 09 18:42:35.217126 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:35.217084 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-22w7k" podStartSLOduration=65.637310982 podStartE2EDuration="1m11.21706716s" totalImagesPullingTime="5.579756178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:42:35.216381882 +0000 UTC m=+71.960364994" watchObservedRunningTime="2026-07-09 18:42:35.21706716 +0000 UTC m=+71.961050271" Jul 09 18:42:35.231552 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:35.231497 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8mts9" podStartSLOduration=65.591626612 podStartE2EDuration="1m11.231479553s" totalImagesPullingTime="5.639852941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:42:35.23123039 +0000 UTC m=+71.975213505" watchObservedRunningTime="2026-07-09 18:42:35.231479553 +0000 UTC m=+71.975462664" Jul 09 18:42:36.808195 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.808161 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz"] Jul 09 18:42:36.814365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.814343 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" Jul 09 18:42:36.822008 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.821962 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 18:42:36.822094 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.821964 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 18:42:36.823908 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.823735 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 09 18:42:36.823908 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.823794 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 18:42:36.837954 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.837921 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz"] Jul 09 18:42:36.857771 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.857748 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-dd6cd8b87-5pbbf"] Jul 09 18:42:36.888264 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.888224 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcqkd\" (UniqueName: \"kubernetes.io/projected/550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9-kube-api-access-xcqkd\") pod \"klusterlet-addon-workmgr-85bb45c4f6-4rqfz\" (UID: \"550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" Jul 09 18:42:36.888442 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.888389 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9-tmp\") pod \"klusterlet-addon-workmgr-85bb45c4f6-4rqfz\" (UID: \"550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" Jul 09 18:42:36.888442 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.888426 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9-klusterlet-config\") pod \"klusterlet-addon-workmgr-85bb45c4f6-4rqfz\" (UID: \"550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" Jul 09 18:42:36.898005 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.897977 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-m8nk7"] Jul 09 18:42:36.902781 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.902760 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-m8nk7" Jul 09 18:42:36.906729 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.906693 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-48bjj\"" Jul 09 18:42:36.906943 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.906929 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 18:42:36.907387 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.907370 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 18:42:36.916510 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.916487 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-m8nk7"] Jul 09 18:42:36.989094 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.989066 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdxgd\" (UniqueName: \"kubernetes.io/projected/68f27912-05a0-4cf9-ab28-e97d87bb5f19-kube-api-access-jdxgd\") pod \"downloads-68c9cbb497-m8nk7\" (UID: \"68f27912-05a0-4cf9-ab28-e97d87bb5f19\") " pod="openshift-console/downloads-68c9cbb497-m8nk7" Jul 09 18:42:36.989260 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.989126 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9-tmp\") pod \"klusterlet-addon-workmgr-85bb45c4f6-4rqfz\" (UID: \"550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" Jul 09 18:42:36.989260 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.989161 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9-klusterlet-config\") pod \"klusterlet-addon-workmgr-85bb45c4f6-4rqfz\" (UID: \"550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" Jul 09 18:42:36.989260 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.989210 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcqkd\" (UniqueName: \"kubernetes.io/projected/550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9-kube-api-access-xcqkd\") pod \"klusterlet-addon-workmgr-85bb45c4f6-4rqfz\" (UID: \"550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" Jul 09 18:42:36.989507 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.989488 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9-tmp\") pod \"klusterlet-addon-workmgr-85bb45c4f6-4rqfz\" (UID: \"550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" Jul 09 18:42:36.992118 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:36.992097 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9-klusterlet-config\") pod \"klusterlet-addon-workmgr-85bb45c4f6-4rqfz\" (UID: \"550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" Jul 09 18:42:37.002541 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:37.002495 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcqkd\" (UniqueName: \"kubernetes.io/projected/550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9-kube-api-access-xcqkd\") pod \"klusterlet-addon-workmgr-85bb45c4f6-4rqfz\" (UID: \"550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" Jul 09 18:42:37.089913 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:37.089815 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdxgd\" (UniqueName: \"kubernetes.io/projected/68f27912-05a0-4cf9-ab28-e97d87bb5f19-kube-api-access-jdxgd\") pod \"downloads-68c9cbb497-m8nk7\" (UID: \"68f27912-05a0-4cf9-ab28-e97d87bb5f19\") " pod="openshift-console/downloads-68c9cbb497-m8nk7" Jul 09 18:42:37.099857 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:37.099829 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdxgd\" (UniqueName: \"kubernetes.io/projected/68f27912-05a0-4cf9-ab28-e97d87bb5f19-kube-api-access-jdxgd\") pod \"downloads-68c9cbb497-m8nk7\" (UID: \"68f27912-05a0-4cf9-ab28-e97d87bb5f19\") " pod="openshift-console/downloads-68c9cbb497-m8nk7" Jul 09 18:42:37.125779 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:37.125744 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" Jul 09 18:42:37.212805 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:37.212774 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-m8nk7" Jul 09 18:42:39.318188 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:39.318166 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-m8nk7"] Jul 09 18:42:39.321664 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:39.321637 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f27912_05a0_4cf9_ab28_e97d87bb5f19.slice/crio-adc9e93fc022d4f2e3b3b1f74c7331afa81030877065db17bbbf412633ecb9e8 WatchSource:0}: Error finding container adc9e93fc022d4f2e3b3b1f74c7331afa81030877065db17bbbf412633ecb9e8: Status 404 returned error can't find the container with id adc9e93fc022d4f2e3b3b1f74c7331afa81030877065db17bbbf412633ecb9e8 Jul 09 18:42:39.333492 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:39.333467 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz"] Jul 09 18:42:39.337503 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:39.337480 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod550e78b8_b8d9_4ddb_af07_e9ffaae3bbb9.slice/crio-b52d120543e9fe4317bb6085b72757a5b207bb1a4e9f273b4cb32b048dba04e0 WatchSource:0}: Error finding container b52d120543e9fe4317bb6085b72757a5b207bb1a4e9f273b4cb32b048dba04e0: Status 404 returned error can't find the container with id b52d120543e9fe4317bb6085b72757a5b207bb1a4e9f273b4cb32b048dba04e0 Jul 09 18:42:40.229511 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:40.229417 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r8dks" event={"ID":"e7275e4d-1d71-4b91-9ae2-a78ef40557e7","Type":"ContainerStarted","Data":"cb60edae18281b548db373d9c35fd2cdc8cbde259ef18eb727e7fa2fb1c006a5"} Jul 09 18:42:40.231256 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:40.231185 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-m8nk7" event={"ID":"68f27912-05a0-4cf9-ab28-e97d87bb5f19","Type":"ContainerStarted","Data":"adc9e93fc022d4f2e3b3b1f74c7331afa81030877065db17bbbf412633ecb9e8"} Jul 09 18:42:40.232731 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:40.232703 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" event={"ID":"550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9","Type":"ContainerStarted","Data":"b52d120543e9fe4317bb6085b72757a5b207bb1a4e9f273b4cb32b048dba04e0"} Jul 09 18:42:40.235290 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:40.235252 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xb7b" event={"ID":"f9ae055f-9c62-4b47-b584-c1941319a5ed","Type":"ContainerStarted","Data":"8f49f8f9d41f2720feb284048fa208ba6ce952aff0ddfb6299f420555c23af18"} Jul 09 18:42:40.235422 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:40.235303 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xb7b" event={"ID":"f9ae055f-9c62-4b47-b584-c1941319a5ed","Type":"ContainerStarted","Data":"eb483b8393090f5cd123c843fca35f09af125d1e68fbf15e01243b2194ccdafb"} Jul 09 18:42:40.235489 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:40.235455 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:41.242928 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:41.242774 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r8dks" event={"ID":"e7275e4d-1d71-4b91-9ae2-a78ef40557e7","Type":"ContainerStarted","Data":"8e12ed158b5d5192bdb657e84717cb7e5ea5d7c4aaced54beba226a16da48d43"} Jul 09 18:42:42.945769 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:42.945724 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qpgdh\" (UID: \"383d00ba-23bb-45b1-a968-71b5dda948c5\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" Jul 09 18:42:42.950695 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:42.950639 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/383d00ba-23bb-45b1-a968-71b5dda948c5-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-qpgdh\" (UID: \"383d00ba-23bb-45b1-a968-71b5dda948c5\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" Jul 09 18:42:43.158574 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:43.158533 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tsqbj\"" Jul 09 18:42:43.166095 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:43.166062 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" Jul 09 18:42:44.929278 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:44.929216 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7xb7b" podStartSLOduration=40.532949812 podStartE2EDuration="44.929190147s" totalImagesPullingTime="4.396240335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:42:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:42:40.254863749 +0000 UTC m=+76.998846862" watchObservedRunningTime="2026-07-09 18:42:44.929190147 +0000 UTC m=+81.673173268" Jul 09 18:42:44.929727 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:44.929605 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-qpgdh"] Jul 09 18:42:44.933224 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:44.933196 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod383d00ba_23bb_45b1_a968_71b5dda948c5.slice/crio-f6c962d208b9232089d475bab1389df204ffe9cddfad97f3f59eebccf2901116 WatchSource:0}: Error finding container f6c962d208b9232089d475bab1389df204ffe9cddfad97f3f59eebccf2901116: Status 404 returned error can't find the container with id f6c962d208b9232089d475bab1389df204ffe9cddfad97f3f59eebccf2901116 Jul 09 18:42:45.268486 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.268446 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r8dks" event={"ID":"e7275e4d-1d71-4b91-9ae2-a78ef40557e7","Type":"ContainerStarted","Data":"3a6070b5c81589c3549bed2e9fc20062837f51bcea7b9ca1ec34179c4f96496e"} Jul 09 18:42:45.270110 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.270075 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" event={"ID":"550e78b8-b8d9-4ddb-af07-e9ffaae3bbb9","Type":"ContainerStarted","Data":"3970eed5f9fce3f865b313953323df8025478103a26f2c2d6afd4686a0948a62"} Jul 09 18:42:45.270233 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.270201 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" Jul 09 18:42:45.271186 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.271162 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" event={"ID":"383d00ba-23bb-45b1-a968-71b5dda948c5","Type":"ContainerStarted","Data":"f6c962d208b9232089d475bab1389df204ffe9cddfad97f3f59eebccf2901116"} Jul 09 18:42:45.272288 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.272265 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" Jul 09 18:42:45.290200 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.290147 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-r8dks" podStartSLOduration=20.476606566 podStartE2EDuration="30.290131717s" totalImagesPullingTime="9.813525151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:42:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:42:45.288473948 +0000 UTC m=+82.032457063" watchObservedRunningTime="2026-07-09 18:42:45.290131717 +0000 UTC m=+82.034114831" Jul 09 18:42:45.306884 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.306831 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85bb45c4f6-4rqfz" podStartSLOduration=3.857293092 podStartE2EDuration="9.306813663s" totalImagesPullingTime="5.449520571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:42:45.305865042 +0000 UTC m=+82.049848170" watchObservedRunningTime="2026-07-09 18:42:45.306813663 +0000 UTC m=+82.050796779" Jul 09 18:42:45.827297 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.827258 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78d695ccf7-rbxt5"] Jul 09 18:42:45.830744 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.830721 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:45.834152 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.833455 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 18:42:45.834304 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.834282 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 18:42:45.834372 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.834307 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 18:42:45.834372 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.834335 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 18:42:45.834431 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.834293 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bvkvd\"" Jul 09 18:42:45.834431 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.834415 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 18:42:45.844178 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.844141 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78d695ccf7-rbxt5"] Jul 09 18:42:45.972777 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.972465 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8ca4f72-486c-4d99-a44a-709990c24049-console-serving-cert\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:45.972777 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.972519 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8ca4f72-486c-4d99-a44a-709990c24049-console-oauth-config\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:45.972777 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.972552 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-console-config\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:45.972777 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.972570 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-oauth-serving-cert\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:45.972777 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.972620 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpgdb\" (UniqueName: \"kubernetes.io/projected/e8ca4f72-486c-4d99-a44a-709990c24049-kube-api-access-qpgdb\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:45.972777 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:45.972661 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-service-ca\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:46.073737 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:46.073695 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8ca4f72-486c-4d99-a44a-709990c24049-console-serving-cert\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:46.073902 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:46.073765 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8ca4f72-486c-4d99-a44a-709990c24049-console-oauth-config\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:46.073902 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:46.073817 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-console-config\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:46.073902 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:46.073850 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-oauth-serving-cert\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:46.073902 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:46.073889 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpgdb\" (UniqueName: \"kubernetes.io/projected/e8ca4f72-486c-4d99-a44a-709990c24049-kube-api-access-qpgdb\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:46.074065 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:46.073937 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-service-ca\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:46.074727 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:46.074693 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-service-ca\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:46.075317 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:46.075289 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-oauth-serving-cert\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:46.075666 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:46.075644 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-console-config\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:46.077666 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:46.077616 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8ca4f72-486c-4d99-a44a-709990c24049-console-oauth-config\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:46.077739 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:46.077724 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8ca4f72-486c-4d99-a44a-709990c24049-console-serving-cert\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:46.085897 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:46.085851 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpgdb\" (UniqueName: \"kubernetes.io/projected/e8ca4f72-486c-4d99-a44a-709990c24049-kube-api-access-qpgdb\") pod \"console-78d695ccf7-rbxt5\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:46.144611 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:46.144496 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:46.395291 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:46.395267 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78d695ccf7-rbxt5"] Jul 09 18:42:46.398061 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:46.398034 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8ca4f72_486c_4d99_a44a_709990c24049.slice/crio-1f3a19ca8a822ab916f3ee7560dff99885be5b83d8a4cc8243fec2fc88d070d2 WatchSource:0}: Error finding container 1f3a19ca8a822ab916f3ee7560dff99885be5b83d8a4cc8243fec2fc88d070d2: Status 404 returned error can't find the container with id 1f3a19ca8a822ab916f3ee7560dff99885be5b83d8a4cc8243fec2fc88d070d2 Jul 09 18:42:46.865147 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:46.865113 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:42:47.281933 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:47.281898 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d695ccf7-rbxt5" event={"ID":"e8ca4f72-486c-4d99-a44a-709990c24049","Type":"ContainerStarted","Data":"1f3a19ca8a822ab916f3ee7560dff99885be5b83d8a4cc8243fec2fc88d070d2"} Jul 09 18:42:47.284281 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:47.284252 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" event={"ID":"383d00ba-23bb-45b1-a968-71b5dda948c5","Type":"ContainerStarted","Data":"c3546b5226c6bcd9e1d92e2ce4d0e160f08090c499a1161bd013c535ea8ab93e"} Jul 09 18:42:47.317647 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:47.317570 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-qpgdh" podStartSLOduration=35.958652963 podStartE2EDuration="37.317550823s" totalImagesPullingTime="1.35889786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:42:47.316880129 +0000 UTC m=+84.060863244" watchObservedRunningTime="2026-07-09 18:42:47.317550823 +0000 UTC m=+84.061533939" Jul 09 18:42:48.947709 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:48.946306 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-bvdkj"] Jul 09 18:42:48.954003 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:48.953973 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:48.958324 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:48.954677 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-67vv4"] Jul 09 18:42:48.962935 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:48.960102 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-p8bzb\"" Jul 09 18:42:48.963352 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:48.963335 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 09 18:42:48.963818 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:48.963618 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 09 18:42:48.964438 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:48.964361 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:48.965188 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:48.964765 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 09 18:42:48.969514 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:48.969487 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 18:42:48.969861 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:48.969824 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-72qx8\"" Jul 09 18:42:48.971712 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:48.971694 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 18:42:48.972843 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:48.972824 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 18:42:48.975031 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:48.974887 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 18:42:48.976011 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:48.975991 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-bvdkj"] Jul 09 18:42:49.101191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.100557 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.101191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.100626 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99dccb5e-972b-49fe-9fff-8204eabb4335-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.101191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.100658 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-node-exporter-textfile\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.101191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.100682 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/99dccb5e-972b-49fe-9fff-8204eabb4335-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.101191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.100713 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-sys\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.101191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.100736 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/99dccb5e-972b-49fe-9fff-8204eabb4335-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.101191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.100779 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf652\" (UniqueName: \"kubernetes.io/projected/99dccb5e-972b-49fe-9fff-8204eabb4335-kube-api-access-gf652\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.101191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.100812 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-node-exporter-tls\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.101191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.100847 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-node-exporter-accelerators-collector-config\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.101191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.100877 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-node-exporter-wtmp\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.101191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.100903 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-metrics-client-ca\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.101191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.100925 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdr62\" (UniqueName: \"kubernetes.io/projected/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-kube-api-access-bdr62\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.101191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.100957 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-root\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.101191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.100990 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/99dccb5e-972b-49fe-9fff-8204eabb4335-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.101191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.101019 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99dccb5e-972b-49fe-9fff-8204eabb4335-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.202327 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.202243 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf652\" (UniqueName: \"kubernetes.io/projected/99dccb5e-972b-49fe-9fff-8204eabb4335-kube-api-access-gf652\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.202327 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.202309 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-node-exporter-tls\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.202509 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.202347 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-node-exporter-accelerators-collector-config\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.202509 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.202381 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-node-exporter-wtmp\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.202509 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.202405 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-metrics-client-ca\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.202509 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.202429 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdr62\" (UniqueName: \"kubernetes.io/projected/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-kube-api-access-bdr62\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.202509 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.202483 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-root\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.202694 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.202517 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/99dccb5e-972b-49fe-9fff-8204eabb4335-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.202694 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.202548 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99dccb5e-972b-49fe-9fff-8204eabb4335-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.202694 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.202579 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.202694 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.202624 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99dccb5e-972b-49fe-9fff-8204eabb4335-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.202694 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.202652 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-node-exporter-textfile\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.202694 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.202677 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/99dccb5e-972b-49fe-9fff-8204eabb4335-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.202915 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.202707 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-sys\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.202915 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.202731 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/99dccb5e-972b-49fe-9fff-8204eabb4335-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.204399 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.203215 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-root\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.204399 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.204000 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/99dccb5e-972b-49fe-9fff-8204eabb4335-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.204399 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.204124 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99dccb5e-972b-49fe-9fff-8204eabb4335-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.204399 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.204186 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-sys\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.204399 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.204310 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-node-exporter-wtmp\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.204399 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.204361 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/99dccb5e-972b-49fe-9fff-8204eabb4335-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.204994 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.204859 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-node-exporter-accelerators-collector-config\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.205165 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.205117 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-node-exporter-textfile\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.205324 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.205287 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-metrics-client-ca\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.207342 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.207313 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/99dccb5e-972b-49fe-9fff-8204eabb4335-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.214711 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.207538 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.214711 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.207943 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-node-exporter-tls\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.216518 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.216466 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99dccb5e-972b-49fe-9fff-8204eabb4335-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.217112 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.217064 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdr62\" (UniqueName: \"kubernetes.io/projected/9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6-kube-api-access-bdr62\") pod \"node-exporter-67vv4\" (UID: \"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6\") " pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:49.220397 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.220353 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf652\" (UniqueName: \"kubernetes.io/projected/99dccb5e-972b-49fe-9fff-8204eabb4335-kube-api-access-gf652\") pod \"kube-state-metrics-785778ff74-bvdkj\" (UID: \"99dccb5e-972b-49fe-9fff-8204eabb4335\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.272448 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.272155 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" Jul 09 18:42:49.285051 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:49.285021 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-67vv4" Jul 09 18:42:50.027565 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.027530 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 18:42:50.042533 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.042498 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.045237 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.045204 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 18:42:50.045495 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.045478 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 18:42:50.045728 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.045701 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tdfjh\"" Jul 09 18:42:50.046122 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.046102 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 18:42:50.046206 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.046174 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 18:42:50.046367 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.046349 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 18:42:50.047848 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.046700 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 18:42:50.047848 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.046884 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 18:42:50.047848 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.047245 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 18:42:50.047848 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.047390 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 18:42:50.047848 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.047576 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 18:42:50.110799 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.110730 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.110799 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.110781 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.111049 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.110808 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.111049 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.110897 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-config-volume\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.111049 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.110945 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3badd9e0-b6f1-434c-85e0-19e315ae4122-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.111049 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.110975 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3badd9e0-b6f1-434c-85e0-19e315ae4122-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.111049 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.111018 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.111049 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.111042 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3badd9e0-b6f1-434c-85e0-19e315ae4122-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.111333 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.111069 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3badd9e0-b6f1-434c-85e0-19e315ae4122-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.111333 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.111145 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rj88\" (UniqueName: \"kubernetes.io/projected/3badd9e0-b6f1-434c-85e0-19e315ae4122-kube-api-access-4rj88\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.111333 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.111182 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3badd9e0-b6f1-434c-85e0-19e315ae4122-config-out\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.111333 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.111235 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.111333 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.111273 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-web-config\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.188793 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:50.188668 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b5d8f55_7a0e_4bbc_ac2e_2914a88d44e6.slice/crio-f99ce8d97133ffc65a9e8287be8a1db348bad2461a9e73f8b9ce0f26696d9b5c WatchSource:0}: Error finding container f99ce8d97133ffc65a9e8287be8a1db348bad2461a9e73f8b9ce0f26696d9b5c: Status 404 returned error can't find the container with id f99ce8d97133ffc65a9e8287be8a1db348bad2461a9e73f8b9ce0f26696d9b5c Jul 09 18:42:50.212219 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.212183 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3badd9e0-b6f1-434c-85e0-19e315ae4122-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.212219 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.212239 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rj88\" (UniqueName: \"kubernetes.io/projected/3badd9e0-b6f1-434c-85e0-19e315ae4122-kube-api-access-4rj88\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.212474 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.212270 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3badd9e0-b6f1-434c-85e0-19e315ae4122-config-out\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.212474 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.212322 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.212474 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.212355 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-web-config\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.212474 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.212381 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.212474 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.212421 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.212474 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.212448 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.212800 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.212485 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-config-volume\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.212800 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.212515 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3badd9e0-b6f1-434c-85e0-19e315ae4122-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.212800 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.212544 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3badd9e0-b6f1-434c-85e0-19e315ae4122-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.212800 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.212593 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.212800 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.212619 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3badd9e0-b6f1-434c-85e0-19e315ae4122-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.213375 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.213352 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3badd9e0-b6f1-434c-85e0-19e315ae4122-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.215504 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.213915 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3badd9e0-b6f1-434c-85e0-19e315ae4122-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.215504 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.215197 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3badd9e0-b6f1-434c-85e0-19e315ae4122-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.216888 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.216419 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.216888 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.216842 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.218108 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.218068 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.218782 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.218740 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.218959 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.218942 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3badd9e0-b6f1-434c-85e0-19e315ae4122-config-out\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.219529 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.219484 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3badd9e0-b6f1-434c-85e0-19e315ae4122-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.219839 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.219818 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-config-volume\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.220356 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.220331 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.220818 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.220784 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-web-config\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.224067 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.224029 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rj88\" (UniqueName: \"kubernetes.io/projected/3badd9e0-b6f1-434c-85e0-19e315ae4122-kube-api-access-4rj88\") pod \"alertmanager-main-0\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.247134 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.246446 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7xb7b" Jul 09 18:42:50.304123 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.304073 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-67vv4" event={"ID":"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6","Type":"ContainerStarted","Data":"f99ce8d97133ffc65a9e8287be8a1db348bad2461a9e73f8b9ce0f26696d9b5c"} Jul 09 18:42:50.344822 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.344791 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-bvdkj"] Jul 09 18:42:50.347358 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:50.347316 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99dccb5e_972b_49fe_9fff_8204eabb4335.slice/crio-0052928f375ca57266b26a1cc1747ff564a6160d7d4c1f7ac9f1eb7d6c1e7f45 WatchSource:0}: Error finding container 0052928f375ca57266b26a1cc1747ff564a6160d7d4c1f7ac9f1eb7d6c1e7f45: Status 404 returned error can't find the container with id 0052928f375ca57266b26a1cc1747ff564a6160d7d4c1f7ac9f1eb7d6c1e7f45 Jul 09 18:42:50.364596 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.364366 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:42:50.537680 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:50.537655 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 18:42:50.540658 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:42:50.540627 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3badd9e0_b6f1_434c_85e0_19e315ae4122.slice/crio-02fecc36ec51853f2b1dd8a786da09c029e8e819b37319e000f66ba2fa063693 WatchSource:0}: Error finding container 02fecc36ec51853f2b1dd8a786da09c029e8e819b37319e000f66ba2fa063693: Status 404 returned error can't find the container with id 02fecc36ec51853f2b1dd8a786da09c029e8e819b37319e000f66ba2fa063693 Jul 09 18:42:51.312238 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:51.312144 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-67vv4" event={"ID":"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6","Type":"ContainerStarted","Data":"ca8af9471b9e0cbc88df58361d6da3b803097d8c25a212640762b0fb77d1c957"} Jul 09 18:42:51.315138 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:51.315107 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" event={"ID":"99dccb5e-972b-49fe-9fff-8204eabb4335","Type":"ContainerStarted","Data":"0052928f375ca57266b26a1cc1747ff564a6160d7d4c1f7ac9f1eb7d6c1e7f45"} Jul 09 18:42:51.318082 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:51.317987 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d695ccf7-rbxt5" event={"ID":"e8ca4f72-486c-4d99-a44a-709990c24049","Type":"ContainerStarted","Data":"41c7cca037d58a9f7d70d8a968d37c56006dfc78d609e1eea4be440a4f2f9931"} Jul 09 18:42:51.320890 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:51.320798 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3badd9e0-b6f1-434c-85e0-19e315ae4122","Type":"ContainerStarted","Data":"02fecc36ec51853f2b1dd8a786da09c029e8e819b37319e000f66ba2fa063693"} Jul 09 18:42:51.366453 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:51.366390 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78d695ccf7-rbxt5" podStartSLOduration=2.5203442430000003 podStartE2EDuration="6.366368724s" totalImagesPullingTime="3.846024481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:42:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:42:51.364796469 +0000 UTC m=+88.108779585" watchObservedRunningTime="2026-07-09 18:42:51.366368724 +0000 UTC m=+88.110351833" Jul 09 18:42:52.327096 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:52.327061 2585 generic.go:356] "Generic (PLEG): container finished" podID="9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6" containerID="ca8af9471b9e0cbc88df58361d6da3b803097d8c25a212640762b0fb77d1c957" exitCode=0 Jul 09 18:42:52.327757 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:52.327729 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-67vv4" event={"ID":"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6","Type":"ContainerDied","Data":"ca8af9471b9e0cbc88df58361d6da3b803097d8c25a212640762b0fb77d1c957"} Jul 09 18:42:53.702000 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:53.701962 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q"] Jul 09 18:42:53.707568 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:53.707543 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q" Jul 09 18:42:53.710728 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:53.710703 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-t6dkk\"" Jul 09 18:42:53.710846 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:53.710738 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 09 18:42:53.716629 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:53.716221 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q"] Jul 09 18:42:53.846601 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:53.846530 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/613b7cd8-3e1c-4ab1-805f-769e799a8a27-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-sqg4q\" (UID: \"613b7cd8-3e1c-4ab1-805f-769e799a8a27\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q" Jul 09 18:42:53.846792 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:53.846688 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxt2f\" (UniqueName: \"kubernetes.io/projected/613b7cd8-3e1c-4ab1-805f-769e799a8a27-kube-api-access-hxt2f\") pod \"monitoring-plugin-5f68df7f58-sqg4q\" (UID: \"613b7cd8-3e1c-4ab1-805f-769e799a8a27\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q" Jul 09 18:42:53.948025 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:53.947989 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/613b7cd8-3e1c-4ab1-805f-769e799a8a27-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-sqg4q\" (UID: \"613b7cd8-3e1c-4ab1-805f-769e799a8a27\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q" Jul 09 18:42:53.948191 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:53.948172 2585 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 09 18:42:53.948255 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:42:53.948246 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/613b7cd8-3e1c-4ab1-805f-769e799a8a27-monitoring-plugin-cert podName:613b7cd8-3e1c-4ab1-805f-769e799a8a27 nodeName:}" failed. No retries permitted until 2026-07-09 18:42:54.448225869 +0000 UTC m=+91.192208978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/613b7cd8-3e1c-4ab1-805f-769e799a8a27-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-sqg4q" (UID: "613b7cd8-3e1c-4ab1-805f-769e799a8a27") : secret "monitoring-plugin-cert" not found Jul 09 18:42:53.948391 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:53.948363 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxt2f\" (UniqueName: \"kubernetes.io/projected/613b7cd8-3e1c-4ab1-805f-769e799a8a27-kube-api-access-hxt2f\") pod \"monitoring-plugin-5f68df7f58-sqg4q\" (UID: \"613b7cd8-3e1c-4ab1-805f-769e799a8a27\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q" Jul 09 18:42:53.958013 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:53.957948 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxt2f\" (UniqueName: \"kubernetes.io/projected/613b7cd8-3e1c-4ab1-805f-769e799a8a27-kube-api-access-hxt2f\") pod \"monitoring-plugin-5f68df7f58-sqg4q\" (UID: \"613b7cd8-3e1c-4ab1-805f-769e799a8a27\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q" Jul 09 18:42:54.420641 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.420604 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8494766cf9-tk9pl"] Jul 09 18:42:54.432946 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.432918 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.434129 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.434103 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8494766cf9-tk9pl"] Jul 09 18:42:54.441329 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.441300 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 18:42:54.451919 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.451893 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/613b7cd8-3e1c-4ab1-805f-769e799a8a27-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-sqg4q\" (UID: \"613b7cd8-3e1c-4ab1-805f-769e799a8a27\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q" Jul 09 18:42:54.454625 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.454573 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/613b7cd8-3e1c-4ab1-805f-769e799a8a27-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-sqg4q\" (UID: \"613b7cd8-3e1c-4ab1-805f-769e799a8a27\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q" Jul 09 18:42:54.552884 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.552844 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8a70296-4f70-40f0-9026-68939e749712-console-serving-cert\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.552884 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.552889 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-trusted-ca-bundle\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.553158 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.552908 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl4qm\" (UniqueName: \"kubernetes.io/projected/d8a70296-4f70-40f0-9026-68939e749712-kube-api-access-pl4qm\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.553158 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.553049 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-service-ca\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.553158 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.553116 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8a70296-4f70-40f0-9026-68939e749712-console-oauth-config\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.553297 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.553161 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-console-config\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.553297 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.553206 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-oauth-serving-cert\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.621201 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.621164 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q" Jul 09 18:42:54.654128 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.654093 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-service-ca\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.654287 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.654161 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8a70296-4f70-40f0-9026-68939e749712-console-oauth-config\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.654287 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.654207 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-console-config\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.654287 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.654244 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-oauth-serving-cert\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.654453 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.654299 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8a70296-4f70-40f0-9026-68939e749712-console-serving-cert\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.654453 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.654324 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-trusted-ca-bundle\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.654453 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.654352 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl4qm\" (UniqueName: \"kubernetes.io/projected/d8a70296-4f70-40f0-9026-68939e749712-kube-api-access-pl4qm\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.654806 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.654777 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-service-ca\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.655331 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.655310 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-oauth-serving-cert\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.655723 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.655696 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-console-config\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.655839 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.655819 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-trusted-ca-bundle\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.657453 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.657428 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8a70296-4f70-40f0-9026-68939e749712-console-oauth-config\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.657837 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.657815 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8a70296-4f70-40f0-9026-68939e749712-console-serving-cert\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.664093 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.664060 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl4qm\" (UniqueName: \"kubernetes.io/projected/d8a70296-4f70-40f0-9026-68939e749712-kube-api-access-pl4qm\") pod \"console-8494766cf9-tk9pl\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:54.746816 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:54.746727 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:42:55.175769 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.175739 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:42:55.216881 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.216822 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:42:55.217028 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.216970 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.222194 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.221936 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wxcxw\"" Jul 09 18:42:55.222194 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.221986 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 18:42:55.222381 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.222351 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 18:42:55.222381 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.222364 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 18:42:55.222645 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.222626 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 18:42:55.223169 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.223141 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 18:42:55.228485 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.228004 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 18:42:55.228485 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.228048 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 18:42:55.228485 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.228094 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 18:42:55.228485 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.228004 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 18:42:55.228485 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.228229 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 18:42:55.228485 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.228342 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 18:42:55.228485 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.228395 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b8503crt8iqhi\"" Jul 09 18:42:55.229527 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.228933 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 18:42:55.230764 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.230743 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 18:42:55.232113 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.232094 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 18:42:55.241554 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.241537 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 18:42:55.360740 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.360708 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.360913 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.360785 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.360913 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.360811 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.360913 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.360829 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.360913 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.360855 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.360913 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.360884 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.360913 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.360904 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.361100 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.360923 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwsq9\" (UniqueName: \"kubernetes.io/projected/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-kube-api-access-wwsq9\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.361100 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.360941 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-config\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.361100 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.360959 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.361100 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.360973 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.361100 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.361013 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.361100 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.361038 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.361100 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.361056 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.361100 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.361075 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.361100 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.361090 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.361351 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.361105 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.361351 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.361120 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.361351 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.361141 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.361351 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.361156 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462033 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.461942 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462033 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.461996 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462033 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462023 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462319 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462050 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462319 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462079 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462319 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462107 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462319 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462132 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462319 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462159 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwsq9\" (UniqueName: \"kubernetes.io/projected/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-kube-api-access-wwsq9\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462319 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462189 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-config\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462319 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462216 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462319 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462240 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462319 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462298 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462332 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462362 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462385 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462409 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462440 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462466 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.462813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462757 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.463142 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.462991 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.463142 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.463028 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.463142 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.463030 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.463696 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.463571 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.463976 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.463857 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.464744 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.464715 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.464840 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.464811 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.466635 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.465836 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.466635 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.465993 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.466635 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.466010 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.466844 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.466824 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.468170 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.467665 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.468170 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.467755 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.468170 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.468129 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.468680 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.468620 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.469016 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.468833 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.469390 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.469352 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-config\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.472253 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.469727 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.472253 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.469829 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.472253 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.471957 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.477177 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.477156 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwsq9\" (UniqueName: \"kubernetes.io/projected/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-kube-api-access-wwsq9\") pod \"prometheus-k8s-0\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:55.529156 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:55.529114 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:42:56.145495 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:56.145463 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:56.146106 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:56.145525 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:56.152066 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:56.152041 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:42:56.346818 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:42:56.346785 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:43:00.093162 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.093072 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8494766cf9-tk9pl"] Jul 09 18:43:00.098749 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:43:00.098706 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8a70296_4f70_40f0_9026_68939e749712.slice/crio-da68557f9d7aa2514b13574efbcdce1c5211bfb81e68cd94e36d3642b07faf06 WatchSource:0}: Error finding container da68557f9d7aa2514b13574efbcdce1c5211bfb81e68cd94e36d3642b07faf06: Status 404 returned error can't find the container with id da68557f9d7aa2514b13574efbcdce1c5211bfb81e68cd94e36d3642b07faf06 Jul 09 18:43:00.120232 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.119320 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q"] Jul 09 18:43:00.163340 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:43:00.163305 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod613b7cd8_3e1c_4ab1_805f_769e799a8a27.slice/crio-53ece703d73cda88f16e790e178a487845d7dd8b27ba1cd325fe42645c76a867 WatchSource:0}: Error finding container 53ece703d73cda88f16e790e178a487845d7dd8b27ba1cd325fe42645c76a867: Status 404 returned error can't find the container with id 53ece703d73cda88f16e790e178a487845d7dd8b27ba1cd325fe42645c76a867 Jul 09 18:43:00.169116 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.169022 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:43:00.180216 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:43:00.180189 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6f5e9c9_f9ce_4bb4_aebb_db2f55e452b4.slice/crio-d7f50fe522c9e61d385c481bcb7cb07aa664b80fcd92c8cabc559273d4d1ae35 WatchSource:0}: Error finding container d7f50fe522c9e61d385c481bcb7cb07aa664b80fcd92c8cabc559273d4d1ae35: Status 404 returned error can't find the container with id d7f50fe522c9e61d385c481bcb7cb07aa664b80fcd92c8cabc559273d4d1ae35 Jul 09 18:43:00.360237 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.360198 2585 generic.go:356] "Generic (PLEG): container finished" podID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerID="dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458" exitCode=0 Jul 09 18:43:00.360387 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.360299 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3badd9e0-b6f1-434c-85e0-19e315ae4122","Type":"ContainerDied","Data":"dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458"} Jul 09 18:43:00.362860 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.362832 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-m8nk7" event={"ID":"68f27912-05a0-4cf9-ab28-e97d87bb5f19","Type":"ContainerStarted","Data":"2e6d1b2188e7e8d39e42515ef559c206ae3313375e11de20cb1e8d572fb87501"} Jul 09 18:43:00.363139 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.363094 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-m8nk7" Jul 09 18:43:00.365772 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.365744 2585 patch_prober.go:28] interesting pod/downloads-68c9cbb497-m8nk7 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.15:8080/\": dial tcp 10.133.0.15:8080: connect: connection refused" start-of-body= Jul 09 18:43:00.365852 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.365795 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-68c9cbb497-m8nk7" podUID="68f27912-05a0-4cf9-ab28-e97d87bb5f19" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.15:8080/\": dial tcp 10.133.0.15:8080: connect: connection refused" Jul 09 18:43:00.366722 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.366695 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8494766cf9-tk9pl" event={"ID":"d8a70296-4f70-40f0-9026-68939e749712","Type":"ContainerStarted","Data":"5d82a45aabb825ff557a4c5539641510fae727e0224cb22018b2f7fb6d163cf0"} Jul 09 18:43:00.366802 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.366745 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8494766cf9-tk9pl" event={"ID":"d8a70296-4f70-40f0-9026-68939e749712","Type":"ContainerStarted","Data":"da68557f9d7aa2514b13574efbcdce1c5211bfb81e68cd94e36d3642b07faf06"} Jul 09 18:43:00.373836 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.373812 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-67vv4" event={"ID":"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6","Type":"ContainerStarted","Data":"c89fb577f0397bac72562bba1231ff3a7f91436bfdea8030ebac3e990e534ee6"} Jul 09 18:43:00.373914 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.373865 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-67vv4" event={"ID":"9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6","Type":"ContainerStarted","Data":"56ea614f071161ae415b4dcba1b9dccbbe7d60776d22b7b5e9f7faf0e85248d3"} Jul 09 18:43:00.376331 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.376258 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q" event={"ID":"613b7cd8-3e1c-4ab1-805f-769e799a8a27","Type":"ContainerStarted","Data":"53ece703d73cda88f16e790e178a487845d7dd8b27ba1cd325fe42645c76a867"} Jul 09 18:43:00.378363 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.378338 2585 generic.go:356] "Generic (PLEG): container finished" podID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerID="75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f" exitCode=0 Jul 09 18:43:00.378438 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.378421 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4","Type":"ContainerDied","Data":"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f"} Jul 09 18:43:00.378496 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.378481 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4","Type":"ContainerStarted","Data":"d7f50fe522c9e61d385c481bcb7cb07aa664b80fcd92c8cabc559273d4d1ae35"} Jul 09 18:43:00.383543 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.383431 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" event={"ID":"99dccb5e-972b-49fe-9fff-8204eabb4335","Type":"ContainerStarted","Data":"33c7ab2a2b1eeed4ceef12838e24a226b0eae0c9f0056f31e5b9add576d84f66"} Jul 09 18:43:00.383543 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.383480 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" event={"ID":"99dccb5e-972b-49fe-9fff-8204eabb4335","Type":"ContainerStarted","Data":"93bc993b3fc76fe472372cbdc9d4ab112982073a5aa18124f1282346de68a132"} Jul 09 18:43:00.383543 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.383509 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" event={"ID":"99dccb5e-972b-49fe-9fff-8204eabb4335","Type":"ContainerStarted","Data":"224ff9508efc13c91906f7b2cd63185c6185385305acf2bdc55f24f86c7fe73a"} Jul 09 18:43:00.418525 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.418482 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-bvdkj" podStartSLOduration=2.898521195 podStartE2EDuration="12.418459059s" totalImagesPullingTime="9.519937864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:42:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:43:00.418027858 +0000 UTC m=+97.162011000" watchObservedRunningTime="2026-07-09 18:43:00.418459059 +0000 UTC m=+97.162442173" Jul 09 18:43:00.482060 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.481962 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-67vv4" podStartSLOduration=11.395676175 podStartE2EDuration="12.48194731s" totalImagesPullingTime="1.000290703s" totalInitContainerRuntime="85.980432ms" isStatefulPod=true podCreationTimestamp="2026-07-09 18:42:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:43:00.480400685 +0000 UTC m=+97.224383800" watchObservedRunningTime="2026-07-09 18:43:00.48194731 +0000 UTC m=+97.225930424" Jul 09 18:43:00.501646 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.501563 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-m8nk7" podStartSLOduration=3.885526081 podStartE2EDuration="24.501541239s" totalImagesPullingTime="20.616015158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:43:00.500858163 +0000 UTC m=+97.244841278" watchObservedRunningTime="2026-07-09 18:43:00.501541239 +0000 UTC m=+97.245524354" Jul 09 18:43:00.523281 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:00.523227 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8494766cf9-tk9pl" podStartSLOduration=6.523206218 podStartE2EDuration="6.523206218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:42:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:43:00.521512439 +0000 UTC m=+97.265495554" watchObservedRunningTime="2026-07-09 18:43:00.523206218 +0000 UTC m=+97.267189334" Jul 09 18:43:01.404725 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:01.404699 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-m8nk7" Jul 09 18:43:01.885878 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:01.885769 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" podUID="027f8931-9686-4794-af63-3f3662f5aba5" containerName="registry" containerID="cri-o://5636c328839b80f349d6facb44adbc1e7c355da88956f9384c1929e122cce502" gracePeriod=30 Jul 09 18:43:02.058777 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.058334 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8494766cf9-tk9pl"] Jul 09 18:43:02.396043 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.396005 2585 generic.go:356] "Generic (PLEG): container finished" podID="027f8931-9686-4794-af63-3f3662f5aba5" containerID="5636c328839b80f349d6facb44adbc1e7c355da88956f9384c1929e122cce502" exitCode=0 Jul 09 18:43:02.396365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.396339 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" event={"ID":"027f8931-9686-4794-af63-3f3662f5aba5","Type":"ContainerDied","Data":"5636c328839b80f349d6facb44adbc1e7c355da88956f9384c1929e122cce502"} Jul 09 18:43:02.843262 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.843229 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:43:02.944886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.944059 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njnq4\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-kube-api-access-njnq4\") pod \"027f8931-9686-4794-af63-3f3662f5aba5\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " Jul 09 18:43:02.944886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.944135 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/027f8931-9686-4794-af63-3f3662f5aba5-image-registry-private-configuration\") pod \"027f8931-9686-4794-af63-3f3662f5aba5\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " Jul 09 18:43:02.944886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.944168 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-bound-sa-token\") pod \"027f8931-9686-4794-af63-3f3662f5aba5\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " Jul 09 18:43:02.944886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.944228 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/027f8931-9686-4794-af63-3f3662f5aba5-registry-certificates\") pod \"027f8931-9686-4794-af63-3f3662f5aba5\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " Jul 09 18:43:02.944886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.944276 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/027f8931-9686-4794-af63-3f3662f5aba5-ca-trust-extracted\") pod \"027f8931-9686-4794-af63-3f3662f5aba5\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " Jul 09 18:43:02.944886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.944316 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls\") pod \"027f8931-9686-4794-af63-3f3662f5aba5\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " Jul 09 18:43:02.944886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.944347 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/027f8931-9686-4794-af63-3f3662f5aba5-installation-pull-secrets\") pod \"027f8931-9686-4794-af63-3f3662f5aba5\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " Jul 09 18:43:02.944886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.944380 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/027f8931-9686-4794-af63-3f3662f5aba5-trusted-ca\") pod \"027f8931-9686-4794-af63-3f3662f5aba5\" (UID: \"027f8931-9686-4794-af63-3f3662f5aba5\") " Jul 09 18:43:02.946011 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.945814 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/027f8931-9686-4794-af63-3f3662f5aba5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "027f8931-9686-4794-af63-3f3662f5aba5" (UID: "027f8931-9686-4794-af63-3f3662f5aba5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:43:02.946011 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.945850 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/027f8931-9686-4794-af63-3f3662f5aba5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "027f8931-9686-4794-af63-3f3662f5aba5" (UID: "027f8931-9686-4794-af63-3f3662f5aba5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:43:02.956394 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.956367 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/027f8931-9686-4794-af63-3f3662f5aba5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "027f8931-9686-4794-af63-3f3662f5aba5" (UID: "027f8931-9686-4794-af63-3f3662f5aba5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:43:02.956952 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.956784 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/027f8931-9686-4794-af63-3f3662f5aba5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "027f8931-9686-4794-af63-3f3662f5aba5" (UID: "027f8931-9686-4794-af63-3f3662f5aba5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:43:02.965260 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.965211 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/027f8931-9686-4794-af63-3f3662f5aba5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "027f8931-9686-4794-af63-3f3662f5aba5" (UID: "027f8931-9686-4794-af63-3f3662f5aba5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:43:02.965440 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.965399 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "027f8931-9686-4794-af63-3f3662f5aba5" (UID: "027f8931-9686-4794-af63-3f3662f5aba5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:43:02.965811 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.965773 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-kube-api-access-njnq4" (OuterVolumeSpecName: "kube-api-access-njnq4") pod "027f8931-9686-4794-af63-3f3662f5aba5" (UID: "027f8931-9686-4794-af63-3f3662f5aba5"). InnerVolumeSpecName "kube-api-access-njnq4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:43:02.965918 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:02.965862 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "027f8931-9686-4794-af63-3f3662f5aba5" (UID: "027f8931-9686-4794-af63-3f3662f5aba5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:43:03.046612 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:03.046504 2585 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/027f8931-9686-4794-af63-3f3662f5aba5-registry-certificates\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:03.046612 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:03.046543 2585 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/027f8931-9686-4794-af63-3f3662f5aba5-ca-trust-extracted\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:03.046612 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:03.046558 2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-registry-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:03.046612 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:03.046570 2585 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/027f8931-9686-4794-af63-3f3662f5aba5-installation-pull-secrets\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:03.046612 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:03.046578 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/027f8931-9686-4794-af63-3f3662f5aba5-trusted-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:03.046612 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:03.046617 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njnq4\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-kube-api-access-njnq4\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:03.046982 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:03.046631 2585 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/027f8931-9686-4794-af63-3f3662f5aba5-image-registry-private-configuration\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:03.046982 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:03.046645 2585 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/027f8931-9686-4794-af63-3f3662f5aba5-bound-sa-token\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:03.403140 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:03.403050 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" event={"ID":"027f8931-9686-4794-af63-3f3662f5aba5","Type":"ContainerDied","Data":"dc2f643c8ef45d175211e2c120a3276f37664a11a076ff9844a497a1c51ef5b5"} Jul 09 18:43:03.403294 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:03.403153 2585 scope.go:117] "RemoveContainer" containerID="5636c328839b80f349d6facb44adbc1e7c355da88956f9384c1929e122cce502" Jul 09 18:43:03.403364 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:03.403319 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dd6cd8b87-5pbbf" Jul 09 18:43:03.431092 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:03.431037 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-dd6cd8b87-5pbbf"] Jul 09 18:43:03.435501 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:03.435430 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-dd6cd8b87-5pbbf"] Jul 09 18:43:03.845610 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:03.845554 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="027f8931-9686-4794-af63-3f3662f5aba5" path="/var/lib/kubelet/pods/027f8931-9686-4794-af63-3f3662f5aba5/volumes" Jul 09 18:43:04.747545 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:04.747509 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:43:06.211286 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:06.211256 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-22w7k" Jul 09 18:43:06.423573 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:06.423533 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4","Type":"ContainerStarted","Data":"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d"} Jul 09 18:43:06.423801 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:06.423646 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4","Type":"ContainerStarted","Data":"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6"} Jul 09 18:43:06.428276 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:06.428247 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3badd9e0-b6f1-434c-85e0-19e315ae4122","Type":"ContainerStarted","Data":"fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae"} Jul 09 18:43:06.428408 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:06.428306 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3badd9e0-b6f1-434c-85e0-19e315ae4122","Type":"ContainerStarted","Data":"31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db"} Jul 09 18:43:06.428408 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:06.428344 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3badd9e0-b6f1-434c-85e0-19e315ae4122","Type":"ContainerStarted","Data":"69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545"} Jul 09 18:43:06.428408 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:06.428385 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3badd9e0-b6f1-434c-85e0-19e315ae4122","Type":"ContainerStarted","Data":"734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078"} Jul 09 18:43:06.428609 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:06.428415 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3badd9e0-b6f1-434c-85e0-19e315ae4122","Type":"ContainerStarted","Data":"2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7"} Jul 09 18:43:06.430488 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:06.430453 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q" event={"ID":"613b7cd8-3e1c-4ab1-805f-769e799a8a27","Type":"ContainerStarted","Data":"fcec4d20321f8c51af7e4e1189709740791b49337976b8457c709ff8a907008d"} Jul 09 18:43:06.431837 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:06.431780 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q" Jul 09 18:43:06.439155 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:06.439131 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q" Jul 09 18:43:06.472616 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:06.471562 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sqg4q" podStartSLOduration=8.176148678 podStartE2EDuration="13.471541029s" totalImagesPullingTime="5.295392351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:42:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:43:06.452724276 +0000 UTC m=+103.196707388" watchObservedRunningTime="2026-07-09 18:43:06.471541029 +0000 UTC m=+103.215524145" Jul 09 18:43:08.444412 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:08.444376 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3badd9e0-b6f1-434c-85e0-19e315ae4122","Type":"ContainerStarted","Data":"347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450"} Jul 09 18:43:08.480322 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:08.480245 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.611299058 podStartE2EDuration="18.480222871s" totalImagesPullingTime="15.844222226s" totalInitContainerRuntime="24.701587ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:42:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:43:08.478907982 +0000 UTC m=+105.222891108" watchObservedRunningTime="2026-07-09 18:43:08.480222871 +0000 UTC m=+105.224205986" Jul 09 18:43:09.451746 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:09.451706 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4","Type":"ContainerStarted","Data":"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60"} Jul 09 18:43:09.452096 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:09.451804 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4","Type":"ContainerStarted","Data":"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e"} Jul 09 18:43:09.452096 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:09.451832 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4","Type":"ContainerStarted","Data":"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17"} Jul 09 18:43:09.452096 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:09.451875 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4","Type":"ContainerStarted","Data":"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac"} Jul 09 18:43:09.486417 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:09.486357 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.309416442 podStartE2EDuration="14.486338324s" totalImagesPullingTime="8.14851179s" totalInitContainerRuntime="28.410092ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:42:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:43:09.485192706 +0000 UTC m=+106.229175820" watchObservedRunningTime="2026-07-09 18:43:09.486338324 +0000 UTC m=+106.230321437" Jul 09 18:43:10.529244 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:10.529211 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:43:12.732129 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:12.732091 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78d695ccf7-rbxt5"] Jul 09 18:43:27.425566 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.425503 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8494766cf9-tk9pl" podUID="d8a70296-4f70-40f0-9026-68939e749712" containerName="console" containerID="cri-o://5d82a45aabb825ff557a4c5539641510fae727e0224cb22018b2f7fb6d163cf0" gracePeriod=15 Jul 09 18:43:27.700106 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.700026 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:43:27.772215 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.772184 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-service-ca\") pod \"d8a70296-4f70-40f0-9026-68939e749712\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " Jul 09 18:43:27.772394 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.772250 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-oauth-serving-cert\") pod \"d8a70296-4f70-40f0-9026-68939e749712\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " Jul 09 18:43:27.772394 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.772269 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-console-config\") pod \"d8a70296-4f70-40f0-9026-68939e749712\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " Jul 09 18:43:27.772394 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.772378 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8a70296-4f70-40f0-9026-68939e749712-console-oauth-config\") pod \"d8a70296-4f70-40f0-9026-68939e749712\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " Jul 09 18:43:27.772572 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.772443 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8a70296-4f70-40f0-9026-68939e749712-console-serving-cert\") pod \"d8a70296-4f70-40f0-9026-68939e749712\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " Jul 09 18:43:27.772572 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.772481 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pl4qm\" (UniqueName: \"kubernetes.io/projected/d8a70296-4f70-40f0-9026-68939e749712-kube-api-access-pl4qm\") pod \"d8a70296-4f70-40f0-9026-68939e749712\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " Jul 09 18:43:27.772572 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.772506 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-trusted-ca-bundle\") pod \"d8a70296-4f70-40f0-9026-68939e749712\" (UID: \"d8a70296-4f70-40f0-9026-68939e749712\") " Jul 09 18:43:27.772740 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.772696 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-console-config" (OuterVolumeSpecName: "console-config") pod "d8a70296-4f70-40f0-9026-68939e749712" (UID: "d8a70296-4f70-40f0-9026-68939e749712"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:43:27.772740 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.772707 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-service-ca" (OuterVolumeSpecName: "service-ca") pod "d8a70296-4f70-40f0-9026-68939e749712" (UID: "d8a70296-4f70-40f0-9026-68939e749712"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:43:27.772740 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.772714 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d8a70296-4f70-40f0-9026-68939e749712" (UID: "d8a70296-4f70-40f0-9026-68939e749712"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:43:27.772886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.772833 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:27.772886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.772850 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-oauth-serving-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:27.772886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.772862 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-console-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:27.773045 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.773026 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d8a70296-4f70-40f0-9026-68939e749712" (UID: "d8a70296-4f70-40f0-9026-68939e749712"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:43:27.774944 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.774919 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8a70296-4f70-40f0-9026-68939e749712-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d8a70296-4f70-40f0-9026-68939e749712" (UID: "d8a70296-4f70-40f0-9026-68939e749712"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:43:27.775048 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.774968 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8a70296-4f70-40f0-9026-68939e749712-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d8a70296-4f70-40f0-9026-68939e749712" (UID: "d8a70296-4f70-40f0-9026-68939e749712"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:43:27.775048 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.775013 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8a70296-4f70-40f0-9026-68939e749712-kube-api-access-pl4qm" (OuterVolumeSpecName: "kube-api-access-pl4qm") pod "d8a70296-4f70-40f0-9026-68939e749712" (UID: "d8a70296-4f70-40f0-9026-68939e749712"). InnerVolumeSpecName "kube-api-access-pl4qm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:43:27.874147 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.874114 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8a70296-4f70-40f0-9026-68939e749712-console-oauth-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:27.874147 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.874144 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8a70296-4f70-40f0-9026-68939e749712-console-serving-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:27.874306 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.874158 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pl4qm\" (UniqueName: \"kubernetes.io/projected/d8a70296-4f70-40f0-9026-68939e749712-kube-api-access-pl4qm\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:27.874306 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:27.874170 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8a70296-4f70-40f0-9026-68939e749712-trusted-ca-bundle\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:28.528496 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:28.528462 2585 generic.go:356] "Generic (PLEG): container finished" podID="d8a70296-4f70-40f0-9026-68939e749712" containerID="5d82a45aabb825ff557a4c5539641510fae727e0224cb22018b2f7fb6d163cf0" exitCode=0 Jul 09 18:43:28.528922 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:28.528520 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8494766cf9-tk9pl" Jul 09 18:43:28.528922 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:28.528546 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8494766cf9-tk9pl" event={"ID":"d8a70296-4f70-40f0-9026-68939e749712","Type":"ContainerDied","Data":"5d82a45aabb825ff557a4c5539641510fae727e0224cb22018b2f7fb6d163cf0"} Jul 09 18:43:28.528922 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:28.528618 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8494766cf9-tk9pl" event={"ID":"d8a70296-4f70-40f0-9026-68939e749712","Type":"ContainerDied","Data":"da68557f9d7aa2514b13574efbcdce1c5211bfb81e68cd94e36d3642b07faf06"} Jul 09 18:43:28.528922 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:28.528647 2585 scope.go:117] "RemoveContainer" containerID="5d82a45aabb825ff557a4c5539641510fae727e0224cb22018b2f7fb6d163cf0" Jul 09 18:43:28.536915 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:28.536896 2585 scope.go:117] "RemoveContainer" containerID="5d82a45aabb825ff557a4c5539641510fae727e0224cb22018b2f7fb6d163cf0" Jul 09 18:43:28.537191 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:43:28.537172 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d82a45aabb825ff557a4c5539641510fae727e0224cb22018b2f7fb6d163cf0\": container with ID starting with 5d82a45aabb825ff557a4c5539641510fae727e0224cb22018b2f7fb6d163cf0 not found: ID does not exist" containerID="5d82a45aabb825ff557a4c5539641510fae727e0224cb22018b2f7fb6d163cf0" Jul 09 18:43:28.537242 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:28.537202 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d82a45aabb825ff557a4c5539641510fae727e0224cb22018b2f7fb6d163cf0"} err="failed to get container status \"5d82a45aabb825ff557a4c5539641510fae727e0224cb22018b2f7fb6d163cf0\": rpc error: code = NotFound desc = could not find container \"5d82a45aabb825ff557a4c5539641510fae727e0224cb22018b2f7fb6d163cf0\": container with ID starting with 5d82a45aabb825ff557a4c5539641510fae727e0224cb22018b2f7fb6d163cf0 not found: ID does not exist" Jul 09 18:43:28.565719 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:28.565692 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8494766cf9-tk9pl"] Jul 09 18:43:28.580833 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:28.580807 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8494766cf9-tk9pl"] Jul 09 18:43:29.842164 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:29.842134 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8a70296-4f70-40f0-9026-68939e749712" path="/var/lib/kubelet/pods/d8a70296-4f70-40f0-9026-68939e749712/volumes" Jul 09 18:43:35.556828 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:35.556792 2585 generic.go:356] "Generic (PLEG): container finished" podID="424fc5f3-4cd6-4463-85b3-ec78268f6ede" containerID="f1bbb25dd3d3fac6504853037197458196bc0f9be80f47628f99dc7a4de36f85" exitCode=0 Jul 09 18:43:35.556828 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:35.556831 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" event={"ID":"424fc5f3-4cd6-4463-85b3-ec78268f6ede","Type":"ContainerDied","Data":"f1bbb25dd3d3fac6504853037197458196bc0f9be80f47628f99dc7a4de36f85"} Jul 09 18:43:35.557225 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:35.557157 2585 scope.go:117] "RemoveContainer" containerID="f1bbb25dd3d3fac6504853037197458196bc0f9be80f47628f99dc7a4de36f85" Jul 09 18:43:36.567929 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:36.567895 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mhttr" event={"ID":"424fc5f3-4cd6-4463-85b3-ec78268f6ede","Type":"ContainerStarted","Data":"e8c4abec8d1bd8ef84f0dc4ea65236a4521cea68191e7efe0e8b32f3a7e86ba0"} Jul 09 18:43:37.752495 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:37.752445 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78d695ccf7-rbxt5" podUID="e8ca4f72-486c-4d99-a44a-709990c24049" containerName="console" containerID="cri-o://41c7cca037d58a9f7d70d8a968d37c56006dfc78d609e1eea4be440a4f2f9931" gracePeriod=15 Jul 09 18:43:38.016325 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.016269 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:43:38.166438 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.166395 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-console-config\") pod \"e8ca4f72-486c-4d99-a44a-709990c24049\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " Jul 09 18:43:38.166671 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.166467 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-service-ca\") pod \"e8ca4f72-486c-4d99-a44a-709990c24049\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " Jul 09 18:43:38.166671 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.166505 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8ca4f72-486c-4d99-a44a-709990c24049-console-serving-cert\") pod \"e8ca4f72-486c-4d99-a44a-709990c24049\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " Jul 09 18:43:38.166671 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.166554 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8ca4f72-486c-4d99-a44a-709990c24049-console-oauth-config\") pod \"e8ca4f72-486c-4d99-a44a-709990c24049\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " Jul 09 18:43:38.166671 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.166577 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-oauth-serving-cert\") pod \"e8ca4f72-486c-4d99-a44a-709990c24049\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " Jul 09 18:43:38.166887 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.166701 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpgdb\" (UniqueName: \"kubernetes.io/projected/e8ca4f72-486c-4d99-a44a-709990c24049-kube-api-access-qpgdb\") pod \"e8ca4f72-486c-4d99-a44a-709990c24049\" (UID: \"e8ca4f72-486c-4d99-a44a-709990c24049\") " Jul 09 18:43:38.167051 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.167017 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-console-config" (OuterVolumeSpecName: "console-config") pod "e8ca4f72-486c-4d99-a44a-709990c24049" (UID: "e8ca4f72-486c-4d99-a44a-709990c24049"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:43:38.167051 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.167027 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-service-ca" (OuterVolumeSpecName: "service-ca") pod "e8ca4f72-486c-4d99-a44a-709990c24049" (UID: "e8ca4f72-486c-4d99-a44a-709990c24049"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:43:38.167236 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.167098 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e8ca4f72-486c-4d99-a44a-709990c24049" (UID: "e8ca4f72-486c-4d99-a44a-709990c24049"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:43:38.169027 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.168994 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8ca4f72-486c-4d99-a44a-709990c24049-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e8ca4f72-486c-4d99-a44a-709990c24049" (UID: "e8ca4f72-486c-4d99-a44a-709990c24049"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:43:38.169106 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.169024 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8ca4f72-486c-4d99-a44a-709990c24049-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e8ca4f72-486c-4d99-a44a-709990c24049" (UID: "e8ca4f72-486c-4d99-a44a-709990c24049"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:43:38.169155 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.169093 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8ca4f72-486c-4d99-a44a-709990c24049-kube-api-access-qpgdb" (OuterVolumeSpecName: "kube-api-access-qpgdb") pod "e8ca4f72-486c-4d99-a44a-709990c24049" (UID: "e8ca4f72-486c-4d99-a44a-709990c24049"). InnerVolumeSpecName "kube-api-access-qpgdb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:43:38.268128 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.268056 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-console-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:38.268128 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.268080 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:38.268128 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.268088 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8ca4f72-486c-4d99-a44a-709990c24049-console-serving-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:38.268128 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.268098 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8ca4f72-486c-4d99-a44a-709990c24049-console-oauth-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:38.268128 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.268107 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8ca4f72-486c-4d99-a44a-709990c24049-oauth-serving-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:38.268128 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.268115 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpgdb\" (UniqueName: \"kubernetes.io/projected/e8ca4f72-486c-4d99-a44a-709990c24049-kube-api-access-qpgdb\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:43:38.576142 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.576063 2585 generic.go:356] "Generic (PLEG): container finished" podID="e8ca4f72-486c-4d99-a44a-709990c24049" containerID="41c7cca037d58a9f7d70d8a968d37c56006dfc78d609e1eea4be440a4f2f9931" exitCode=0 Jul 09 18:43:38.576142 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.576106 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d695ccf7-rbxt5" Jul 09 18:43:38.576308 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.576153 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d695ccf7-rbxt5" event={"ID":"e8ca4f72-486c-4d99-a44a-709990c24049","Type":"ContainerDied","Data":"41c7cca037d58a9f7d70d8a968d37c56006dfc78d609e1eea4be440a4f2f9931"} Jul 09 18:43:38.576308 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.576198 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d695ccf7-rbxt5" event={"ID":"e8ca4f72-486c-4d99-a44a-709990c24049","Type":"ContainerDied","Data":"1f3a19ca8a822ab916f3ee7560dff99885be5b83d8a4cc8243fec2fc88d070d2"} Jul 09 18:43:38.576308 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.576218 2585 scope.go:117] "RemoveContainer" containerID="41c7cca037d58a9f7d70d8a968d37c56006dfc78d609e1eea4be440a4f2f9931" Jul 09 18:43:38.584809 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.584793 2585 scope.go:117] "RemoveContainer" containerID="41c7cca037d58a9f7d70d8a968d37c56006dfc78d609e1eea4be440a4f2f9931" Jul 09 18:43:38.585044 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:43:38.585026 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41c7cca037d58a9f7d70d8a968d37c56006dfc78d609e1eea4be440a4f2f9931\": container with ID starting with 41c7cca037d58a9f7d70d8a968d37c56006dfc78d609e1eea4be440a4f2f9931 not found: ID does not exist" containerID="41c7cca037d58a9f7d70d8a968d37c56006dfc78d609e1eea4be440a4f2f9931" Jul 09 18:43:38.585114 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.585054 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41c7cca037d58a9f7d70d8a968d37c56006dfc78d609e1eea4be440a4f2f9931"} err="failed to get container status \"41c7cca037d58a9f7d70d8a968d37c56006dfc78d609e1eea4be440a4f2f9931\": rpc error: code = NotFound desc = could not find container \"41c7cca037d58a9f7d70d8a968d37c56006dfc78d609e1eea4be440a4f2f9931\": container with ID starting with 41c7cca037d58a9f7d70d8a968d37c56006dfc78d609e1eea4be440a4f2f9931 not found: ID does not exist" Jul 09 18:43:38.598487 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.598465 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78d695ccf7-rbxt5"] Jul 09 18:43:38.606740 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:38.606718 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78d695ccf7-rbxt5"] Jul 09 18:43:39.841337 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:39.841306 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8ca4f72-486c-4d99-a44a-709990c24049" path="/var/lib/kubelet/pods/e8ca4f72-486c-4d99-a44a-709990c24049/volumes" Jul 09 18:43:55.529935 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:55.529890 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:43:55.549948 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:55.549921 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:43:55.666256 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:43:55.666231 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:09.346086 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:09.346050 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 18:44:09.346555 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:09.346360 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="alertmanager" containerID="cri-o://2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7" gracePeriod=120 Jul 09 18:44:09.346555 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:09.346428 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="kube-rbac-proxy-metric" containerID="cri-o://fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae" gracePeriod=120 Jul 09 18:44:09.346555 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:09.346466 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="kube-rbac-proxy-web" containerID="cri-o://69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545" gracePeriod=120 Jul 09 18:44:09.346555 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:09.346485 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="config-reloader" containerID="cri-o://734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078" gracePeriod=120 Jul 09 18:44:09.346555 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:09.346536 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="kube-rbac-proxy" containerID="cri-o://31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db" gracePeriod=120 Jul 09 18:44:09.346837 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:09.346493 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="prom-label-proxy" containerID="cri-o://347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450" gracePeriod=120 Jul 09 18:44:09.708091 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:09.708055 2585 generic.go:356] "Generic (PLEG): container finished" podID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerID="347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450" exitCode=0 Jul 09 18:44:09.708091 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:09.708083 2585 generic.go:356] "Generic (PLEG): container finished" podID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerID="31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db" exitCode=0 Jul 09 18:44:09.708091 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:09.708089 2585 generic.go:356] "Generic (PLEG): container finished" podID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerID="734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078" exitCode=0 Jul 09 18:44:09.708091 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:09.708095 2585 generic.go:356] "Generic (PLEG): container finished" podID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerID="2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7" exitCode=0 Jul 09 18:44:09.708371 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:09.708120 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3badd9e0-b6f1-434c-85e0-19e315ae4122","Type":"ContainerDied","Data":"347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450"} Jul 09 18:44:09.708371 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:09.708217 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3badd9e0-b6f1-434c-85e0-19e315ae4122","Type":"ContainerDied","Data":"31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db"} Jul 09 18:44:09.708371 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:09.708248 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3badd9e0-b6f1-434c-85e0-19e315ae4122","Type":"ContainerDied","Data":"734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078"} Jul 09 18:44:09.708371 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:09.708276 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3badd9e0-b6f1-434c-85e0-19e315ae4122","Type":"ContainerDied","Data":"2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7"} Jul 09 18:44:10.595787 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.595761 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:44:10.650979 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.650946 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy-metric\") pod \"3badd9e0-b6f1-434c-85e0-19e315ae4122\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " Jul 09 18:44:10.651149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.651008 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy\") pod \"3badd9e0-b6f1-434c-85e0-19e315ae4122\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " Jul 09 18:44:10.651149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.651043 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3badd9e0-b6f1-434c-85e0-19e315ae4122-config-out\") pod \"3badd9e0-b6f1-434c-85e0-19e315ae4122\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " Jul 09 18:44:10.651149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.651074 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-main-tls\") pod \"3badd9e0-b6f1-434c-85e0-19e315ae4122\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " Jul 09 18:44:10.651149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.651115 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-config-volume\") pod \"3badd9e0-b6f1-434c-85e0-19e315ae4122\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " Jul 09 18:44:10.651365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.651151 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rj88\" (UniqueName: \"kubernetes.io/projected/3badd9e0-b6f1-434c-85e0-19e315ae4122-kube-api-access-4rj88\") pod \"3badd9e0-b6f1-434c-85e0-19e315ae4122\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " Jul 09 18:44:10.651365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.651185 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-web-config\") pod \"3badd9e0-b6f1-434c-85e0-19e315ae4122\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " Jul 09 18:44:10.651365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.651214 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3badd9e0-b6f1-434c-85e0-19e315ae4122-tls-assets\") pod \"3badd9e0-b6f1-434c-85e0-19e315ae4122\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " Jul 09 18:44:10.651365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.651243 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3badd9e0-b6f1-434c-85e0-19e315ae4122-alertmanager-trusted-ca-bundle\") pod \"3badd9e0-b6f1-434c-85e0-19e315ae4122\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " Jul 09 18:44:10.651365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.651271 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy-web\") pod \"3badd9e0-b6f1-434c-85e0-19e315ae4122\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " Jul 09 18:44:10.651365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.651324 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-cluster-tls-config\") pod \"3badd9e0-b6f1-434c-85e0-19e315ae4122\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " Jul 09 18:44:10.651680 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.651380 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3badd9e0-b6f1-434c-85e0-19e315ae4122-alertmanager-main-db\") pod \"3badd9e0-b6f1-434c-85e0-19e315ae4122\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " Jul 09 18:44:10.651680 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.651422 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3badd9e0-b6f1-434c-85e0-19e315ae4122-metrics-client-ca\") pod \"3badd9e0-b6f1-434c-85e0-19e315ae4122\" (UID: \"3badd9e0-b6f1-434c-85e0-19e315ae4122\") " Jul 09 18:44:10.652107 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.652080 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3badd9e0-b6f1-434c-85e0-19e315ae4122-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "3badd9e0-b6f1-434c-85e0-19e315ae4122" (UID: "3badd9e0-b6f1-434c-85e0-19e315ae4122"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:44:10.652256 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.652174 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3badd9e0-b6f1-434c-85e0-19e315ae4122-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "3badd9e0-b6f1-434c-85e0-19e315ae4122" (UID: "3badd9e0-b6f1-434c-85e0-19e315ae4122"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:44:10.652656 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.652633 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3badd9e0-b6f1-434c-85e0-19e315ae4122-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "3badd9e0-b6f1-434c-85e0-19e315ae4122" (UID: "3badd9e0-b6f1-434c-85e0-19e315ae4122"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:44:10.655745 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.655704 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "3badd9e0-b6f1-434c-85e0-19e315ae4122" (UID: "3badd9e0-b6f1-434c-85e0-19e315ae4122"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:10.656309 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.656098 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3badd9e0-b6f1-434c-85e0-19e315ae4122-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3badd9e0-b6f1-434c-85e0-19e315ae4122" (UID: "3badd9e0-b6f1-434c-85e0-19e315ae4122"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:44:10.656309 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.656101 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "3badd9e0-b6f1-434c-85e0-19e315ae4122" (UID: "3badd9e0-b6f1-434c-85e0-19e315ae4122"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:10.656608 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.656546 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "3badd9e0-b6f1-434c-85e0-19e315ae4122" (UID: "3badd9e0-b6f1-434c-85e0-19e315ae4122"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:10.656886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.656857 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "3badd9e0-b6f1-434c-85e0-19e315ae4122" (UID: "3badd9e0-b6f1-434c-85e0-19e315ae4122"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:10.656984 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.656857 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3badd9e0-b6f1-434c-85e0-19e315ae4122-config-out" (OuterVolumeSpecName: "config-out") pod "3badd9e0-b6f1-434c-85e0-19e315ae4122" (UID: "3badd9e0-b6f1-434c-85e0-19e315ae4122"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:44:10.657338 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.657312 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3badd9e0-b6f1-434c-85e0-19e315ae4122-kube-api-access-4rj88" (OuterVolumeSpecName: "kube-api-access-4rj88") pod "3badd9e0-b6f1-434c-85e0-19e315ae4122" (UID: "3badd9e0-b6f1-434c-85e0-19e315ae4122"). InnerVolumeSpecName "kube-api-access-4rj88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:44:10.658688 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.658651 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-config-volume" (OuterVolumeSpecName: "config-volume") pod "3badd9e0-b6f1-434c-85e0-19e315ae4122" (UID: "3badd9e0-b6f1-434c-85e0-19e315ae4122"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:10.661485 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.661458 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "3badd9e0-b6f1-434c-85e0-19e315ae4122" (UID: "3badd9e0-b6f1-434c-85e0-19e315ae4122"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:10.669252 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.669223 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-web-config" (OuterVolumeSpecName: "web-config") pod "3badd9e0-b6f1-434c-85e0-19e315ae4122" (UID: "3badd9e0-b6f1-434c-85e0-19e315ae4122"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:10.715002 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.714969 2585 generic.go:356] "Generic (PLEG): container finished" podID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerID="fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae" exitCode=0 Jul 09 18:44:10.715002 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.714996 2585 generic.go:356] "Generic (PLEG): container finished" podID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerID="69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545" exitCode=0 Jul 09 18:44:10.715201 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.715044 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:44:10.715201 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.715059 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3badd9e0-b6f1-434c-85e0-19e315ae4122","Type":"ContainerDied","Data":"fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae"} Jul 09 18:44:10.715201 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.715133 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3badd9e0-b6f1-434c-85e0-19e315ae4122","Type":"ContainerDied","Data":"69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545"} Jul 09 18:44:10.715201 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.715164 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3badd9e0-b6f1-434c-85e0-19e315ae4122","Type":"ContainerDied","Data":"02fecc36ec51853f2b1dd8a786da09c029e8e819b37319e000f66ba2fa063693"} Jul 09 18:44:10.715328 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.715201 2585 scope.go:117] "RemoveContainer" containerID="347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450" Jul 09 18:44:10.723782 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.723762 2585 scope.go:117] "RemoveContainer" containerID="fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae" Jul 09 18:44:10.730974 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.730955 2585 scope.go:117] "RemoveContainer" containerID="31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db" Jul 09 18:44:10.738227 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.738204 2585 scope.go:117] "RemoveContainer" containerID="69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545" Jul 09 18:44:10.741117 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.741089 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 18:44:10.747265 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.747240 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 18:44:10.749425 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.748023 2585 scope.go:117] "RemoveContainer" containerID="734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078" Jul 09 18:44:10.752907 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.752886 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:10.752993 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.752909 2585 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-cluster-tls-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:10.752993 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.752919 2585 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3badd9e0-b6f1-434c-85e0-19e315ae4122-alertmanager-main-db\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:10.752993 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.752928 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3badd9e0-b6f1-434c-85e0-19e315ae4122-metrics-client-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:10.752993 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.752937 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:10.752993 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.752946 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:10.752993 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.752955 2585 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3badd9e0-b6f1-434c-85e0-19e315ae4122-config-out\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:10.752993 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.752965 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-secret-alertmanager-main-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:10.752993 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.752973 2585 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:10.752993 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.752982 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rj88\" (UniqueName: \"kubernetes.io/projected/3badd9e0-b6f1-434c-85e0-19e315ae4122-kube-api-access-4rj88\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:10.753261 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.753009 2585 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3badd9e0-b6f1-434c-85e0-19e315ae4122-web-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:10.753261 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.753019 2585 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3badd9e0-b6f1-434c-85e0-19e315ae4122-tls-assets\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:10.753261 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.753026 2585 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3badd9e0-b6f1-434c-85e0-19e315ae4122-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:10.755965 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.755950 2585 scope.go:117] "RemoveContainer" containerID="2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7" Jul 09 18:44:10.763413 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.763394 2585 scope.go:117] "RemoveContainer" containerID="dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458" Jul 09 18:44:10.770903 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.770874 2585 scope.go:117] "RemoveContainer" containerID="347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450" Jul 09 18:44:10.771193 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:44:10.771176 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450\": container with ID starting with 347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450 not found: ID does not exist" containerID="347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450" Jul 09 18:44:10.771252 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.771203 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450"} err="failed to get container status \"347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450\": rpc error: code = NotFound desc = could not find container \"347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450\": container with ID starting with 347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450 not found: ID does not exist" Jul 09 18:44:10.771252 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.771222 2585 scope.go:117] "RemoveContainer" containerID="fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae" Jul 09 18:44:10.771459 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:44:10.771442 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae\": container with ID starting with fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae not found: ID does not exist" containerID="fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae" Jul 09 18:44:10.771498 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.771467 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae"} err="failed to get container status \"fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae\": rpc error: code = NotFound desc = could not find container \"fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae\": container with ID starting with fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae not found: ID does not exist" Jul 09 18:44:10.771498 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.771482 2585 scope.go:117] "RemoveContainer" containerID="31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db" Jul 09 18:44:10.771729 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:44:10.771714 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db\": container with ID starting with 31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db not found: ID does not exist" containerID="31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db" Jul 09 18:44:10.771775 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.771733 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db"} err="failed to get container status \"31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db\": rpc error: code = NotFound desc = could not find container \"31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db\": container with ID starting with 31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db not found: ID does not exist" Jul 09 18:44:10.771775 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.771745 2585 scope.go:117] "RemoveContainer" containerID="69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545" Jul 09 18:44:10.772016 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:44:10.771996 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545\": container with ID starting with 69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545 not found: ID does not exist" containerID="69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545" Jul 09 18:44:10.772064 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.772034 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545"} err="failed to get container status \"69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545\": rpc error: code = NotFound desc = could not find container \"69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545\": container with ID starting with 69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545 not found: ID does not exist" Jul 09 18:44:10.772064 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.772056 2585 scope.go:117] "RemoveContainer" containerID="734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078" Jul 09 18:44:10.772307 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:44:10.772291 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078\": container with ID starting with 734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078 not found: ID does not exist" containerID="734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078" Jul 09 18:44:10.772366 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.772312 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078"} err="failed to get container status \"734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078\": rpc error: code = NotFound desc = could not find container \"734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078\": container with ID starting with 734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078 not found: ID does not exist" Jul 09 18:44:10.772366 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.772328 2585 scope.go:117] "RemoveContainer" containerID="2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7" Jul 09 18:44:10.772567 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:44:10.772551 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7\": container with ID starting with 2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7 not found: ID does not exist" containerID="2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7" Jul 09 18:44:10.772633 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.772572 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7"} err="failed to get container status \"2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7\": rpc error: code = NotFound desc = could not find container \"2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7\": container with ID starting with 2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7 not found: ID does not exist" Jul 09 18:44:10.772633 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.772610 2585 scope.go:117] "RemoveContainer" containerID="dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458" Jul 09 18:44:10.772857 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:44:10.772840 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458\": container with ID starting with dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458 not found: ID does not exist" containerID="dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458" Jul 09 18:44:10.772906 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.772862 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458"} err="failed to get container status \"dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458\": rpc error: code = NotFound desc = could not find container \"dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458\": container with ID starting with dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458 not found: ID does not exist" Jul 09 18:44:10.772906 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.772876 2585 scope.go:117] "RemoveContainer" containerID="347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450" Jul 09 18:44:10.773090 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.773071 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450"} err="failed to get container status \"347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450\": rpc error: code = NotFound desc = could not find container \"347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450\": container with ID starting with 347fe165ebc1c56c29bf11c5aaadab5b01f895c3ecb5b1c18d20a44b150f1450 not found: ID does not exist" Jul 09 18:44:10.773136 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.773091 2585 scope.go:117] "RemoveContainer" containerID="fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae" Jul 09 18:44:10.773334 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.773317 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae"} err="failed to get container status \"fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae\": rpc error: code = NotFound desc = could not find container \"fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae\": container with ID starting with fe4ec0ad7e9633a8c0c702520cf8c9a7a609cc9c1f8f7abca6a1e48728dd39ae not found: ID does not exist" Jul 09 18:44:10.773376 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.773335 2585 scope.go:117] "RemoveContainer" containerID="31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db" Jul 09 18:44:10.773552 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.773527 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db"} err="failed to get container status \"31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db\": rpc error: code = NotFound desc = could not find container \"31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db\": container with ID starting with 31ed0df0f52856885e1cea6ed22831ef6d6ebba155cec7a17cc1510839ab92db not found: ID does not exist" Jul 09 18:44:10.773608 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.773555 2585 scope.go:117] "RemoveContainer" containerID="69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545" Jul 09 18:44:10.773807 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.773792 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545"} err="failed to get container status \"69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545\": rpc error: code = NotFound desc = could not find container \"69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545\": container with ID starting with 69bf602db3579de530d60e2e0e76bcd7913a78e1a760a34a71a6b273097ec545 not found: ID does not exist" Jul 09 18:44:10.773852 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.773808 2585 scope.go:117] "RemoveContainer" containerID="734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078" Jul 09 18:44:10.774019 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.773997 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078"} err="failed to get container status \"734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078\": rpc error: code = NotFound desc = could not find container \"734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078\": container with ID starting with 734ed3071b573a07f012db707acc6195ffc26a6ebff875f781c223e469d65078 not found: ID does not exist" Jul 09 18:44:10.774061 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.774021 2585 scope.go:117] "RemoveContainer" containerID="2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7" Jul 09 18:44:10.774213 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.774196 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7"} err="failed to get container status \"2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7\": rpc error: code = NotFound desc = could not find container \"2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7\": container with ID starting with 2a3443188268045fb352945c97c8fe59602a6ee536dc21a1f78912b7cd64ebb7 not found: ID does not exist" Jul 09 18:44:10.774261 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.774212 2585 scope.go:117] "RemoveContainer" containerID="dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458" Jul 09 18:44:10.774395 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:10.774381 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458"} err="failed to get container status \"dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458\": rpc error: code = NotFound desc = could not find container \"dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458\": container with ID starting with dc888906cbc7771cc0315780a3ec57c1f95814d2b9954d17a458b3c1b269e458 not found: ID does not exist" Jul 09 18:44:11.841763 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:11.841687 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" path="/var/lib/kubelet/pods/3badd9e0-b6f1-434c-85e0-19e315ae4122/volumes" Jul 09 18:44:13.673551 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.673410 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:44:13.674061 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.673789 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="prometheus" containerID="cri-o://e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6" gracePeriod=600 Jul 09 18:44:13.674061 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.673836 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60" gracePeriod=600 Jul 09 18:44:13.674061 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.673862 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="kube-rbac-proxy-web" containerID="cri-o://64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17" gracePeriod=600 Jul 09 18:44:13.674061 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.673885 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="thanos-sidecar" containerID="cri-o://e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac" gracePeriod=600 Jul 09 18:44:13.674061 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.673895 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="kube-rbac-proxy" containerID="cri-o://f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e" gracePeriod=600 Jul 09 18:44:13.674061 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.673913 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="config-reloader" containerID="cri-o://4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d" gracePeriod=600 Jul 09 18:44:13.917864 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.917837 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:13.982809 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.982731 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-serving-certs-ca-bundle\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.982809 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.982769 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-metrics-client-certs\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.982809 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.982792 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983068 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.982812 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-web-config\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983068 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.982833 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-tls\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983068 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.982866 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-tls-assets\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983068 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.982887 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-metrics-client-ca\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983068 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.982915 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-kube-rbac-proxy\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983068 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.982944 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwsq9\" (UniqueName: \"kubernetes.io/projected/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-kube-api-access-wwsq9\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983068 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.982970 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-2\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983068 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.983011 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-0\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983068 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.983048 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-db\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983953 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.983076 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-kubelet-serving-ca-bundle\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983953 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.983109 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983953 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.983135 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-grpc-tls\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983953 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.983163 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-config\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983953 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.983198 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-thanos-prometheus-http-client-file\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983953 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.983227 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-trusted-ca-bundle\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983953 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.983275 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-1\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983953 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.983306 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-config-out\") pod \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\" (UID: \"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4\") " Jul 09 18:44:13.983953 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.983196 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:44:13.983953 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.983838 2585 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:13.985193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.984990 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:44:13.985193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.984823 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:44:13.986060 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.985448 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:44:13.986060 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.985779 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:44:13.986336 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.986160 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:44:13.986336 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.986241 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:44:13.987157 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.987051 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:13.987157 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.987128 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:13.987446 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.987416 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:44:13.988177 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.988149 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:13.988820 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.988789 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:13.988912 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.988824 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-config" (OuterVolumeSpecName: "config") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:13.988974 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.988909 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:13.989056 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.989034 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-kube-api-access-wwsq9" (OuterVolumeSpecName: "kube-api-access-wwsq9") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "kube-api-access-wwsq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:44:13.989206 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.989176 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:13.989278 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.989240 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:44:13.989519 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.989492 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-config-out" (OuterVolumeSpecName: "config-out") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:44:13.990843 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.990823 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:13.998936 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:13.998914 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-web-config" (OuterVolumeSpecName: "web-config") pod "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" (UID: "e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:44:14.085102 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085053 2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085102 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085096 2585 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-config-out\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085102 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085106 2585 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-metrics-client-certs\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085102 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085119 2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085129 2585 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-web-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085137 2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085145 2585 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-tls-assets\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085154 2585 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-metrics-client-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085162 2585 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-kube-rbac-proxy\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085170 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwsq9\" (UniqueName: \"kubernetes.io/projected/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-kube-api-access-wwsq9\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085178 2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085186 2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085194 2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-k8s-db\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085203 2585 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085211 2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085219 2585 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-secret-grpc-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085227 2585 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085234 2585 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.085365 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.085242 2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:44:14.738852 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.738821 2585 generic.go:356] "Generic (PLEG): container finished" podID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerID="5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60" exitCode=0 Jul 09 18:44:14.738852 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.738846 2585 generic.go:356] "Generic (PLEG): container finished" podID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerID="f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e" exitCode=0 Jul 09 18:44:14.738852 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.738852 2585 generic.go:356] "Generic (PLEG): container finished" podID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerID="64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17" exitCode=0 Jul 09 18:44:14.738852 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.738857 2585 generic.go:356] "Generic (PLEG): container finished" podID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerID="e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac" exitCode=0 Jul 09 18:44:14.738852 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.738863 2585 generic.go:356] "Generic (PLEG): container finished" podID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerID="4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d" exitCode=0 Jul 09 18:44:14.739386 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.738868 2585 generic.go:356] "Generic (PLEG): container finished" podID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerID="e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6" exitCode=0 Jul 09 18:44:14.739386 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.738901 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.739386 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.738907 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4","Type":"ContainerDied","Data":"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60"} Jul 09 18:44:14.739386 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.738976 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4","Type":"ContainerDied","Data":"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e"} Jul 09 18:44:14.739386 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.739006 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4","Type":"ContainerDied","Data":"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17"} Jul 09 18:44:14.739386 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.739033 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4","Type":"ContainerDied","Data":"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac"} Jul 09 18:44:14.739386 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.739047 2585 scope.go:117] "RemoveContainer" containerID="5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60" Jul 09 18:44:14.739386 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.739074 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4","Type":"ContainerDied","Data":"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d"} Jul 09 18:44:14.739386 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.739117 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4","Type":"ContainerDied","Data":"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6"} Jul 09 18:44:14.739386 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.739157 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4","Type":"ContainerDied","Data":"d7f50fe522c9e61d385c481bcb7cb07aa664b80fcd92c8cabc559273d4d1ae35"} Jul 09 18:44:14.747358 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.747340 2585 scope.go:117] "RemoveContainer" containerID="f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e" Jul 09 18:44:14.754403 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.754383 2585 scope.go:117] "RemoveContainer" containerID="64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17" Jul 09 18:44:14.761075 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.761055 2585 scope.go:117] "RemoveContainer" containerID="e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac" Jul 09 18:44:14.765652 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.765630 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:44:14.768563 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.768542 2585 scope.go:117] "RemoveContainer" containerID="4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d" Jul 09 18:44:14.771275 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.771254 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:44:14.776033 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.776014 2585 scope.go:117] "RemoveContainer" containerID="e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6" Jul 09 18:44:14.783321 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.783306 2585 scope.go:117] "RemoveContainer" containerID="75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f" Jul 09 18:44:14.790238 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.790214 2585 scope.go:117] "RemoveContainer" containerID="5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60" Jul 09 18:44:14.790512 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:44:14.790493 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60\": container with ID starting with 5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60 not found: ID does not exist" containerID="5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60" Jul 09 18:44:14.790559 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.790529 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60"} err="failed to get container status \"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60\": rpc error: code = NotFound desc = could not find container \"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60\": container with ID starting with 5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60 not found: ID does not exist" Jul 09 18:44:14.790559 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.790547 2585 scope.go:117] "RemoveContainer" containerID="f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e" Jul 09 18:44:14.790854 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:44:14.790835 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e\": container with ID starting with f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e not found: ID does not exist" containerID="f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e" Jul 09 18:44:14.790895 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.790862 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e"} err="failed to get container status \"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e\": rpc error: code = NotFound desc = could not find container \"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e\": container with ID starting with f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e not found: ID does not exist" Jul 09 18:44:14.790895 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.790880 2585 scope.go:117] "RemoveContainer" containerID="64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17" Jul 09 18:44:14.791143 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:44:14.791125 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17\": container with ID starting with 64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17 not found: ID does not exist" containerID="64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17" Jul 09 18:44:14.791193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.791147 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17"} err="failed to get container status \"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17\": rpc error: code = NotFound desc = could not find container \"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17\": container with ID starting with 64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17 not found: ID does not exist" Jul 09 18:44:14.791193 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.791163 2585 scope.go:117] "RemoveContainer" containerID="e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac" Jul 09 18:44:14.791373 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:44:14.791359 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac\": container with ID starting with e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac not found: ID does not exist" containerID="e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac" Jul 09 18:44:14.791413 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.791375 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac"} err="failed to get container status \"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac\": rpc error: code = NotFound desc = could not find container \"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac\": container with ID starting with e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac not found: ID does not exist" Jul 09 18:44:14.791413 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.791386 2585 scope.go:117] "RemoveContainer" containerID="4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d" Jul 09 18:44:14.791666 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:44:14.791643 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d\": container with ID starting with 4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d not found: ID does not exist" containerID="4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d" Jul 09 18:44:14.791713 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.791685 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d"} err="failed to get container status \"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d\": rpc error: code = NotFound desc = could not find container \"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d\": container with ID starting with 4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d not found: ID does not exist" Jul 09 18:44:14.791713 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.791705 2585 scope.go:117] "RemoveContainer" containerID="e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6" Jul 09 18:44:14.791933 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:44:14.791918 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6\": container with ID starting with e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6 not found: ID does not exist" containerID="e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6" Jul 09 18:44:14.791982 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.791935 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6"} err="failed to get container status \"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6\": rpc error: code = NotFound desc = could not find container \"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6\": container with ID starting with e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6 not found: ID does not exist" Jul 09 18:44:14.791982 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.791946 2585 scope.go:117] "RemoveContainer" containerID="75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f" Jul 09 18:44:14.792148 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:44:14.792130 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f\": container with ID starting with 75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f not found: ID does not exist" containerID="75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f" Jul 09 18:44:14.792192 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.792165 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f"} err="failed to get container status \"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f\": rpc error: code = NotFound desc = could not find container \"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f\": container with ID starting with 75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f not found: ID does not exist" Jul 09 18:44:14.792192 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.792183 2585 scope.go:117] "RemoveContainer" containerID="5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60" Jul 09 18:44:14.792468 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.792444 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60"} err="failed to get container status \"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60\": rpc error: code = NotFound desc = could not find container \"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60\": container with ID starting with 5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60 not found: ID does not exist" Jul 09 18:44:14.792548 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.792470 2585 scope.go:117] "RemoveContainer" containerID="f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e" Jul 09 18:44:14.792782 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.792761 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e"} err="failed to get container status \"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e\": rpc error: code = NotFound desc = could not find container \"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e\": container with ID starting with f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e not found: ID does not exist" Jul 09 18:44:14.792843 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.792787 2585 scope.go:117] "RemoveContainer" containerID="64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17" Jul 09 18:44:14.793422 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.793401 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17"} err="failed to get container status \"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17\": rpc error: code = NotFound desc = could not find container \"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17\": container with ID starting with 64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17 not found: ID does not exist" Jul 09 18:44:14.793483 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.793422 2585 scope.go:117] "RemoveContainer" containerID="e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac" Jul 09 18:44:14.793697 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.793680 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac"} err="failed to get container status \"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac\": rpc error: code = NotFound desc = could not find container \"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac\": container with ID starting with e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac not found: ID does not exist" Jul 09 18:44:14.793745 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.793698 2585 scope.go:117] "RemoveContainer" containerID="4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d" Jul 09 18:44:14.793925 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.793907 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d"} err="failed to get container status \"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d\": rpc error: code = NotFound desc = could not find container \"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d\": container with ID starting with 4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d not found: ID does not exist" Jul 09 18:44:14.793970 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.793927 2585 scope.go:117] "RemoveContainer" containerID="e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6" Jul 09 18:44:14.794138 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.794121 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6"} err="failed to get container status \"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6\": rpc error: code = NotFound desc = could not find container \"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6\": container with ID starting with e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6 not found: ID does not exist" Jul 09 18:44:14.794176 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.794140 2585 scope.go:117] "RemoveContainer" containerID="75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f" Jul 09 18:44:14.794350 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.794333 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f"} err="failed to get container status \"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f\": rpc error: code = NotFound desc = could not find container \"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f\": container with ID starting with 75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f not found: ID does not exist" Jul 09 18:44:14.794391 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.794350 2585 scope.go:117] "RemoveContainer" containerID="5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60" Jul 09 18:44:14.794578 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.794560 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60"} err="failed to get container status \"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60\": rpc error: code = NotFound desc = could not find container \"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60\": container with ID starting with 5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60 not found: ID does not exist" Jul 09 18:44:14.794705 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.794658 2585 scope.go:117] "RemoveContainer" containerID="f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e" Jul 09 18:44:14.794895 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.794879 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e"} err="failed to get container status \"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e\": rpc error: code = NotFound desc = could not find container \"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e\": container with ID starting with f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e not found: ID does not exist" Jul 09 18:44:14.794939 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.794896 2585 scope.go:117] "RemoveContainer" containerID="64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17" Jul 09 18:44:14.795122 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.795105 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17"} err="failed to get container status \"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17\": rpc error: code = NotFound desc = could not find container \"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17\": container with ID starting with 64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17 not found: ID does not exist" Jul 09 18:44:14.795166 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.795123 2585 scope.go:117] "RemoveContainer" containerID="e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac" Jul 09 18:44:14.795367 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.795351 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac"} err="failed to get container status \"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac\": rpc error: code = NotFound desc = could not find container \"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac\": container with ID starting with e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac not found: ID does not exist" Jul 09 18:44:14.795417 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.795367 2585 scope.go:117] "RemoveContainer" containerID="4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d" Jul 09 18:44:14.795625 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.795609 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d"} err="failed to get container status \"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d\": rpc error: code = NotFound desc = could not find container \"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d\": container with ID starting with 4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d not found: ID does not exist" Jul 09 18:44:14.795662 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.795626 2585 scope.go:117] "RemoveContainer" containerID="e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6" Jul 09 18:44:14.795844 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.795830 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6"} err="failed to get container status \"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6\": rpc error: code = NotFound desc = could not find container \"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6\": container with ID starting with e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6 not found: ID does not exist" Jul 09 18:44:14.795887 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.795845 2585 scope.go:117] "RemoveContainer" containerID="75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f" Jul 09 18:44:14.796064 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.796048 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f"} err="failed to get container status \"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f\": rpc error: code = NotFound desc = could not find container \"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f\": container with ID starting with 75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f not found: ID does not exist" Jul 09 18:44:14.796102 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.796065 2585 scope.go:117] "RemoveContainer" containerID="5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60" Jul 09 18:44:14.796303 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.796285 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60"} err="failed to get container status \"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60\": rpc error: code = NotFound desc = could not find container \"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60\": container with ID starting with 5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60 not found: ID does not exist" Jul 09 18:44:14.796347 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.796304 2585 scope.go:117] "RemoveContainer" containerID="f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e" Jul 09 18:44:14.796552 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.796524 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e"} err="failed to get container status \"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e\": rpc error: code = NotFound desc = could not find container \"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e\": container with ID starting with f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e not found: ID does not exist" Jul 09 18:44:14.796648 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.796554 2585 scope.go:117] "RemoveContainer" containerID="64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17" Jul 09 18:44:14.796806 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.796788 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17"} err="failed to get container status \"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17\": rpc error: code = NotFound desc = could not find container \"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17\": container with ID starting with 64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17 not found: ID does not exist" Jul 09 18:44:14.796848 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.796808 2585 scope.go:117] "RemoveContainer" containerID="e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac" Jul 09 18:44:14.797030 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.797013 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac"} err="failed to get container status \"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac\": rpc error: code = NotFound desc = could not find container \"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac\": container with ID starting with e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac not found: ID does not exist" Jul 09 18:44:14.797070 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.797030 2585 scope.go:117] "RemoveContainer" containerID="4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d" Jul 09 18:44:14.797254 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.797237 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d"} err="failed to get container status \"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d\": rpc error: code = NotFound desc = could not find container \"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d\": container with ID starting with 4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d not found: ID does not exist" Jul 09 18:44:14.797332 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.797256 2585 scope.go:117] "RemoveContainer" containerID="e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6" Jul 09 18:44:14.797519 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.797502 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6"} err="failed to get container status \"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6\": rpc error: code = NotFound desc = could not find container \"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6\": container with ID starting with e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6 not found: ID does not exist" Jul 09 18:44:14.797558 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.797520 2585 scope.go:117] "RemoveContainer" containerID="75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f" Jul 09 18:44:14.797744 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.797728 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f"} err="failed to get container status \"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f\": rpc error: code = NotFound desc = could not find container \"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f\": container with ID starting with 75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f not found: ID does not exist" Jul 09 18:44:14.797818 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.797746 2585 scope.go:117] "RemoveContainer" containerID="5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60" Jul 09 18:44:14.797977 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.797959 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60"} err="failed to get container status \"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60\": rpc error: code = NotFound desc = could not find container \"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60\": container with ID starting with 5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60 not found: ID does not exist" Jul 09 18:44:14.798029 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.797977 2585 scope.go:117] "RemoveContainer" containerID="f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e" Jul 09 18:44:14.798167 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.798141 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e"} err="failed to get container status \"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e\": rpc error: code = NotFound desc = could not find container \"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e\": container with ID starting with f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e not found: ID does not exist" Jul 09 18:44:14.798235 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.798169 2585 scope.go:117] "RemoveContainer" containerID="64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17" Jul 09 18:44:14.798370 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.798354 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17"} err="failed to get container status \"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17\": rpc error: code = NotFound desc = could not find container \"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17\": container with ID starting with 64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17 not found: ID does not exist" Jul 09 18:44:14.798419 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.798370 2585 scope.go:117] "RemoveContainer" containerID="e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac" Jul 09 18:44:14.798551 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.798534 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac"} err="failed to get container status \"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac\": rpc error: code = NotFound desc = could not find container \"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac\": container with ID starting with e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac not found: ID does not exist" Jul 09 18:44:14.798666 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.798551 2585 scope.go:117] "RemoveContainer" containerID="4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d" Jul 09 18:44:14.798775 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.798755 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d"} err="failed to get container status \"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d\": rpc error: code = NotFound desc = could not find container \"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d\": container with ID starting with 4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d not found: ID does not exist" Jul 09 18:44:14.798813 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.798775 2585 scope.go:117] "RemoveContainer" containerID="e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6" Jul 09 18:44:14.798994 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.798977 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6"} err="failed to get container status \"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6\": rpc error: code = NotFound desc = could not find container \"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6\": container with ID starting with e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6 not found: ID does not exist" Jul 09 18:44:14.799046 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.798993 2585 scope.go:117] "RemoveContainer" containerID="75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f" Jul 09 18:44:14.799206 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.799190 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f"} err="failed to get container status \"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f\": rpc error: code = NotFound desc = could not find container \"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f\": container with ID starting with 75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f not found: ID does not exist" Jul 09 18:44:14.799244 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.799207 2585 scope.go:117] "RemoveContainer" containerID="5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60" Jul 09 18:44:14.799400 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.799385 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60"} err="failed to get container status \"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60\": rpc error: code = NotFound desc = could not find container \"5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60\": container with ID starting with 5865add0aafec6705deca8bece1cf7263e9144fef24417eb62f7e1637f6c3b60 not found: ID does not exist" Jul 09 18:44:14.799440 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.799401 2585 scope.go:117] "RemoveContainer" containerID="f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e" Jul 09 18:44:14.799671 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.799654 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e"} err="failed to get container status \"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e\": rpc error: code = NotFound desc = could not find container \"f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e\": container with ID starting with f19371309bebea160791cf621be62ac1579f9fb2477bb9308708a5654a2b377e not found: ID does not exist" Jul 09 18:44:14.799717 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.799671 2585 scope.go:117] "RemoveContainer" containerID="64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17" Jul 09 18:44:14.799863 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.799847 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17"} err="failed to get container status \"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17\": rpc error: code = NotFound desc = could not find container \"64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17\": container with ID starting with 64dba0eccc0fd57fe8b87f5506b4acb5c98c3b585ffc7daf9bcab9aac9c7fd17 not found: ID does not exist" Jul 09 18:44:14.799900 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.799865 2585 scope.go:117] "RemoveContainer" containerID="e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac" Jul 09 18:44:14.800051 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.800035 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac"} err="failed to get container status \"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac\": rpc error: code = NotFound desc = could not find container \"e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac\": container with ID starting with e13b6d62dcc29ed2d5409acdf5bd806bbc132584f759f62db25511273ff0adac not found: ID does not exist" Jul 09 18:44:14.800091 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.800051 2585 scope.go:117] "RemoveContainer" containerID="4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d" Jul 09 18:44:14.800274 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.800257 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d"} err="failed to get container status \"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d\": rpc error: code = NotFound desc = could not find container \"4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d\": container with ID starting with 4000d516eb6f3dcd0ae70436f795bc4a62a544dd789cd0ecee180f1d5df6cb6d not found: ID does not exist" Jul 09 18:44:14.800342 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.800274 2585 scope.go:117] "RemoveContainer" containerID="e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6" Jul 09 18:44:14.800463 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.800443 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6"} err="failed to get container status \"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6\": rpc error: code = NotFound desc = could not find container \"e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6\": container with ID starting with e93c32aafc52623c976f326b615c900f3b12845386202f237d6d0eef497354d6 not found: ID does not exist" Jul 09 18:44:14.800529 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.800465 2585 scope.go:117] "RemoveContainer" containerID="75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f" Jul 09 18:44:14.800741 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.800715 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f"} err="failed to get container status \"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f\": rpc error: code = NotFound desc = could not find container \"75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f\": container with ID starting with 75bc3077d395b095e6a217a02147d85344decee1ba4efeee7864e58228a3234f not found: ID does not exist" Jul 09 18:44:14.802929 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.802905 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:44:14.803210 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803194 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="kube-rbac-proxy-thanos" Jul 09 18:44:14.803287 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803214 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="kube-rbac-proxy-thanos" Jul 09 18:44:14.803287 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803225 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8ca4f72-486c-4d99-a44a-709990c24049" containerName="console" Jul 09 18:44:14.803287 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803233 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8ca4f72-486c-4d99-a44a-709990c24049" containerName="console" Jul 09 18:44:14.803287 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803244 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="init-config-reloader" Jul 09 18:44:14.803287 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803253 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="init-config-reloader" Jul 09 18:44:14.803287 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803266 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="kube-rbac-proxy-web" Jul 09 18:44:14.803287 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803274 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="kube-rbac-proxy-web" Jul 09 18:44:14.803287 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803286 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="alertmanager" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803294 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="alertmanager" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803303 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="config-reloader" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803311 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="config-reloader" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803325 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="027f8931-9686-4794-af63-3f3662f5aba5" containerName="registry" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803333 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="027f8931-9686-4794-af63-3f3662f5aba5" containerName="registry" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803342 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="prometheus" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803351 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="prometheus" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803364 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="kube-rbac-proxy-metric" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803372 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="kube-rbac-proxy-metric" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803382 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="kube-rbac-proxy" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803390 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="kube-rbac-proxy" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803404 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="init-config-reloader" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803412 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="init-config-reloader" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803423 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="config-reloader" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803431 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="config-reloader" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803439 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="prom-label-proxy" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803447 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="prom-label-proxy" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803458 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8a70296-4f70-40f0-9026-68939e749712" containerName="console" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803466 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8a70296-4f70-40f0-9026-68939e749712" containerName="console" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803475 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="thanos-sidecar" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803483 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="thanos-sidecar" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803492 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="kube-rbac-proxy" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803500 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="kube-rbac-proxy" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803515 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="kube-rbac-proxy-web" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803523 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="kube-rbac-proxy-web" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803609 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="027f8931-9686-4794-af63-3f3662f5aba5" containerName="registry" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803620 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="prometheus" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803628 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="kube-rbac-proxy" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803637 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="thanos-sidecar" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803646 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="kube-rbac-proxy" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803654 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="kube-rbac-proxy-web" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803662 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="prom-label-proxy" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803670 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="alertmanager" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803676 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="kube-rbac-proxy-metric" Jul 09 18:44:14.803669 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803683 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="kube-rbac-proxy-thanos" Jul 09 18:44:14.805251 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803692 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8a70296-4f70-40f0-9026-68939e749712" containerName="console" Jul 09 18:44:14.805251 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803700 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" containerName="config-reloader" Jul 09 18:44:14.805251 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803709 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="config-reloader" Jul 09 18:44:14.805251 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803716 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8ca4f72-486c-4d99-a44a-709990c24049" containerName="console" Jul 09 18:44:14.805251 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.803723 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3badd9e0-b6f1-434c-85e0-19e315ae4122" containerName="kube-rbac-proxy-web" Jul 09 18:44:14.808975 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.808958 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.811659 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.811634 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b8503crt8iqhi\"" Jul 09 18:44:14.811747 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.811684 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 18:44:14.811747 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.811721 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 18:44:14.812141 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.812120 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wxcxw\"" Jul 09 18:44:14.812214 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.812162 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 18:44:14.812973 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.812690 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 18:44:14.812973 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.812708 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 18:44:14.812973 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.812692 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 18:44:14.812973 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.812786 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 18:44:14.812973 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.812795 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 18:44:14.812973 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.812830 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 18:44:14.812973 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.812842 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 18:44:14.812973 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.812789 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 18:44:14.812973 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.812938 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 18:44:14.813335 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.812989 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 18:44:14.816415 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.816397 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 18:44:14.818428 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.818411 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 18:44:14.828004 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.827977 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:44:14.890077 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890038 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/50021af2-6813-4fb0-b11a-d70fdecdc0ed-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890077 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890077 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890288 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890093 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm9wx\" (UniqueName: \"kubernetes.io/projected/50021af2-6813-4fb0-b11a-d70fdecdc0ed-kube-api-access-jm9wx\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890288 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890115 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890288 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890183 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890288 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890246 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890288 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890274 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890477 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890294 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890477 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890312 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890477 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890327 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890477 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890357 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890477 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890373 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-config\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890477 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890389 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890477 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890403 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890477 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890420 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890477 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890454 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/50021af2-6813-4fb0-b11a-d70fdecdc0ed-config-out\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890892 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890514 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-web-config\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890892 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890560 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890892 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890623 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/50021af2-6813-4fb0-b11a-d70fdecdc0ed-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.890892 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.890655 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991440 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991343 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/50021af2-6813-4fb0-b11a-d70fdecdc0ed-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991440 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991388 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991667 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991506 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/50021af2-6813-4fb0-b11a-d70fdecdc0ed-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991667 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991539 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991667 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991561 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm9wx\" (UniqueName: \"kubernetes.io/projected/50021af2-6813-4fb0-b11a-d70fdecdc0ed-kube-api-access-jm9wx\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991667 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991623 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991667 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991653 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991705 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991733 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991761 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991783 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991805 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991830 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991854 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-config\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991876 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.991901 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991899 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.992467 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991925 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.992467 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991954 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/50021af2-6813-4fb0-b11a-d70fdecdc0ed-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.992467 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.991964 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/50021af2-6813-4fb0-b11a-d70fdecdc0ed-config-out\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.992467 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.992038 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-web-config\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.992467 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.992082 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.992467 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.992230 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.992810 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.992510 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.992864 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.992812 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.993926 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.993416 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.993926 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.993625 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.994731 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.994683 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.995139 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.995112 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/50021af2-6813-4fb0-b11a-d70fdecdc0ed-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.995533 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.995506 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.995533 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.995523 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.996330 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.996306 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-web-config\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.996786 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.996758 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-config\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.997172 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.997150 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.997310 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.997288 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.997390 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.997347 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/50021af2-6813-4fb0-b11a-d70fdecdc0ed-config-out\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.997438 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.997386 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.997833 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.997818 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.998286 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.998259 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50021af2-6813-4fb0-b11a-d70fdecdc0ed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:14.998370 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:14.998330 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/50021af2-6813-4fb0-b11a-d70fdecdc0ed-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:15.000943 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:15.000926 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm9wx\" (UniqueName: \"kubernetes.io/projected/50021af2-6813-4fb0-b11a-d70fdecdc0ed-kube-api-access-jm9wx\") pod \"prometheus-k8s-0\" (UID: \"50021af2-6813-4fb0-b11a-d70fdecdc0ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:15.119559 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:15.119514 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:44:15.272836 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:15.272781 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:44:15.275353 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:44:15.275327 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50021af2_6813_4fb0_b11a_d70fdecdc0ed.slice/crio-16fc44fb1fbacb5c8a2303bcf53d388418de06b11b9786b0004e3e442d60cfa9 WatchSource:0}: Error finding container 16fc44fb1fbacb5c8a2303bcf53d388418de06b11b9786b0004e3e442d60cfa9: Status 404 returned error can't find the container with id 16fc44fb1fbacb5c8a2303bcf53d388418de06b11b9786b0004e3e442d60cfa9 Jul 09 18:44:15.744435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:15.744399 2585 generic.go:356] "Generic (PLEG): container finished" podID="50021af2-6813-4fb0-b11a-d70fdecdc0ed" containerID="3a9662ffb7caa61eeb77d013a5e236bf075d59413f86ce14a1d83a1dab0be403" exitCode=0 Jul 09 18:44:15.744822 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:15.744445 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50021af2-6813-4fb0-b11a-d70fdecdc0ed","Type":"ContainerDied","Data":"3a9662ffb7caa61eeb77d013a5e236bf075d59413f86ce14a1d83a1dab0be403"} Jul 09 18:44:15.744822 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:15.744557 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50021af2-6813-4fb0-b11a-d70fdecdc0ed","Type":"ContainerStarted","Data":"16fc44fb1fbacb5c8a2303bcf53d388418de06b11b9786b0004e3e442d60cfa9"} Jul 09 18:44:15.840943 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:15.840913 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4" path="/var/lib/kubelet/pods/e6f5e9c9-f9ce-4bb4-aebb-db2f55e452b4/volumes" Jul 09 18:44:16.751513 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:16.751476 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50021af2-6813-4fb0-b11a-d70fdecdc0ed","Type":"ContainerStarted","Data":"a2d83b5cf3495c52e4714c0eba933e0f6aeb050ab0e2e777e41957dc58e89ef6"} Jul 09 18:44:16.752012 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:16.751551 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50021af2-6813-4fb0-b11a-d70fdecdc0ed","Type":"ContainerStarted","Data":"28494d4f532384a0f8bb9924a946c03a1beffd9bab12c95100ddd595c77fac60"} Jul 09 18:44:16.752012 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:16.751620 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50021af2-6813-4fb0-b11a-d70fdecdc0ed","Type":"ContainerStarted","Data":"da1a2df04caccae7f29b1e3d5768dd532f5134a5f24488c4693f50101c323674"} Jul 09 18:44:16.752012 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:16.751664 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50021af2-6813-4fb0-b11a-d70fdecdc0ed","Type":"ContainerStarted","Data":"6323490201afd964ca9fde8b8801d189bd96104640fe3080c4c6c9366bfe939e"} Jul 09 18:44:16.752012 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:16.751707 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50021af2-6813-4fb0-b11a-d70fdecdc0ed","Type":"ContainerStarted","Data":"a64e3d70f957f450e8a29a995ac255bcede8abdcc4b98bbf6323342ff6497f6e"} Jul 09 18:44:16.752012 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:16.751743 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50021af2-6813-4fb0-b11a-d70fdecdc0ed","Type":"ContainerStarted","Data":"25a27777746b113d53583bb6681e71cd99cf860b18fdbec409364b2f783ecb30"} Jul 09 18:44:16.785752 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:16.785673 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.768992869 podStartE2EDuration="2.78565491s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.662041ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:44:16.78389769 +0000 UTC m=+173.527880817" watchObservedRunningTime="2026-07-09 18:44:16.78565491 +0000 UTC m=+173.529638024" Jul 09 18:44:20.120440 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:44:20.120391 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:45:15.120538 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:15.120492 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:45:15.139085 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:15.139059 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:45:15.976561 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:15.976533 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:45:44.693726 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:44.693636 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6hk79"] Jul 09 18:45:44.696877 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:44.696854 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hk79" Jul 09 18:45:44.699257 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:44.699236 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 18:45:44.700191 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:44.700174 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-p9pbj\"" Jul 09 18:45:44.700288 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:44.700215 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 18:45:44.711278 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:44.711251 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6hk79"] Jul 09 18:45:44.734023 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:44.733997 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91f0b9d5-15be-48ed-a3c0-dc7e228046ac-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6hk79\" (UID: \"91f0b9d5-15be-48ed-a3c0-dc7e228046ac\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hk79" Jul 09 18:45:44.734131 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:44.734060 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p7cs\" (UniqueName: \"kubernetes.io/projected/91f0b9d5-15be-48ed-a3c0-dc7e228046ac-kube-api-access-7p7cs\") pod \"cert-manager-cainjector-559cdd8c87-6hk79\" (UID: \"91f0b9d5-15be-48ed-a3c0-dc7e228046ac\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hk79" Jul 09 18:45:44.835272 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:44.835240 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91f0b9d5-15be-48ed-a3c0-dc7e228046ac-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6hk79\" (UID: \"91f0b9d5-15be-48ed-a3c0-dc7e228046ac\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hk79" Jul 09 18:45:44.835396 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:44.835285 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p7cs\" (UniqueName: \"kubernetes.io/projected/91f0b9d5-15be-48ed-a3c0-dc7e228046ac-kube-api-access-7p7cs\") pod \"cert-manager-cainjector-559cdd8c87-6hk79\" (UID: \"91f0b9d5-15be-48ed-a3c0-dc7e228046ac\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hk79" Jul 09 18:45:44.844369 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:44.844347 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p7cs\" (UniqueName: \"kubernetes.io/projected/91f0b9d5-15be-48ed-a3c0-dc7e228046ac-kube-api-access-7p7cs\") pod \"cert-manager-cainjector-559cdd8c87-6hk79\" (UID: \"91f0b9d5-15be-48ed-a3c0-dc7e228046ac\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hk79" Jul 09 18:45:44.844493 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:44.844372 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91f0b9d5-15be-48ed-a3c0-dc7e228046ac-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6hk79\" (UID: \"91f0b9d5-15be-48ed-a3c0-dc7e228046ac\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hk79" Jul 09 18:45:45.014803 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:45.014724 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hk79" Jul 09 18:45:45.142712 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:45.142682 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6hk79"] Jul 09 18:45:45.145824 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:45:45.145791 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91f0b9d5_15be_48ed_a3c0_dc7e228046ac.slice/crio-f8f220a19fe3a1e7f7fef44da11e399264df5de936c41a8eced8241d167c9b27 WatchSource:0}: Error finding container f8f220a19fe3a1e7f7fef44da11e399264df5de936c41a8eced8241d167c9b27: Status 404 returned error can't find the container with id f8f220a19fe3a1e7f7fef44da11e399264df5de936c41a8eced8241d167c9b27 Jul 09 18:45:46.067749 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:46.067707 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hk79" event={"ID":"91f0b9d5-15be-48ed-a3c0-dc7e228046ac","Type":"ContainerStarted","Data":"f8f220a19fe3a1e7f7fef44da11e399264df5de936c41a8eced8241d167c9b27"} Jul 09 18:45:50.087604 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:50.087544 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hk79" event={"ID":"91f0b9d5-15be-48ed-a3c0-dc7e228046ac","Type":"ContainerStarted","Data":"10d19228b2fad0e1a29f7099da1b2d97e4420b31029be1135a5677fcf0054f41"} Jul 09 18:45:50.108285 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:50.108221 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hk79" podStartSLOduration=1.791562632 podStartE2EDuration="6.108198888s" totalImagesPullingTime="4.316636256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:50.105565219 +0000 UTC m=+266.849548335" watchObservedRunningTime="2026-07-09 18:45:50.108198888 +0000 UTC m=+266.852182003" Jul 09 18:45:55.165888 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:55.165856 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rjjvt"] Jul 09 18:45:55.169145 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:55.169129 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rjjvt" Jul 09 18:45:55.171471 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:55.171454 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 18:45:55.171572 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:55.171556 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 18:45:55.172399 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:55.172384 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-zb8x7\"" Jul 09 18:45:55.177698 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:55.177674 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rjjvt"] Jul 09 18:45:55.231363 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:55.231326 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9p2w\" (UniqueName: \"kubernetes.io/projected/b7085015-f3cd-45c6-9c9b-273914615ecb-kube-api-access-t9p2w\") pod \"openshift-lws-operator-fd8ccff4c-rjjvt\" (UID: \"b7085015-f3cd-45c6-9c9b-273914615ecb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rjjvt" Jul 09 18:45:55.231556 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:55.231426 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7085015-f3cd-45c6-9c9b-273914615ecb-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rjjvt\" (UID: \"b7085015-f3cd-45c6-9c9b-273914615ecb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rjjvt" Jul 09 18:45:55.332392 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:55.332364 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7085015-f3cd-45c6-9c9b-273914615ecb-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rjjvt\" (UID: \"b7085015-f3cd-45c6-9c9b-273914615ecb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rjjvt" Jul 09 18:45:55.332573 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:55.332424 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9p2w\" (UniqueName: \"kubernetes.io/projected/b7085015-f3cd-45c6-9c9b-273914615ecb-kube-api-access-t9p2w\") pod \"openshift-lws-operator-fd8ccff4c-rjjvt\" (UID: \"b7085015-f3cd-45c6-9c9b-273914615ecb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rjjvt" Jul 09 18:45:55.332828 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:55.332809 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7085015-f3cd-45c6-9c9b-273914615ecb-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rjjvt\" (UID: \"b7085015-f3cd-45c6-9c9b-273914615ecb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rjjvt" Jul 09 18:45:55.342355 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:55.342327 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9p2w\" (UniqueName: \"kubernetes.io/projected/b7085015-f3cd-45c6-9c9b-273914615ecb-kube-api-access-t9p2w\") pod \"openshift-lws-operator-fd8ccff4c-rjjvt\" (UID: \"b7085015-f3cd-45c6-9c9b-273914615ecb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rjjvt" Jul 09 18:45:55.478272 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:55.478241 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rjjvt" Jul 09 18:45:55.608177 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:55.608153 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rjjvt"] Jul 09 18:45:55.610704 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:45:55.610674 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7085015_f3cd_45c6_9c9b_273914615ecb.slice/crio-6e740f1a5f5b34a5a72d5cd3e86e0f3faea4ad3788854ebc00744a45913ee609 WatchSource:0}: Error finding container 6e740f1a5f5b34a5a72d5cd3e86e0f3faea4ad3788854ebc00744a45913ee609: Status 404 returned error can't find the container with id 6e740f1a5f5b34a5a72d5cd3e86e0f3faea4ad3788854ebc00744a45913ee609 Jul 09 18:45:56.110579 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:56.110543 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rjjvt" event={"ID":"b7085015-f3cd-45c6-9c9b-273914615ecb","Type":"ContainerStarted","Data":"6e740f1a5f5b34a5a72d5cd3e86e0f3faea4ad3788854ebc00744a45913ee609"} Jul 09 18:45:59.124085 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:59.123988 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rjjvt" event={"ID":"b7085015-f3cd-45c6-9c9b-273914615ecb","Type":"ContainerStarted","Data":"18a1281a677051489d693ab96a236356c44dccd590291196d73bf2454f530494"} Jul 09 18:45:59.144017 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:45:59.143966 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rjjvt" podStartSLOduration=1.077868501 podStartE2EDuration="4.143950443s" totalImagesPullingTime="3.066081942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:45:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:59.142769509 +0000 UTC m=+275.886752624" watchObservedRunningTime="2026-07-09 18:45:59.143950443 +0000 UTC m=+275.887933556" Jul 09 18:46:20.906464 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:20.906434 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc"] Jul 09 18:46:20.912735 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:20.912713 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:20.915095 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:20.915069 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 18:46:20.915577 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:20.915561 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 18:46:20.915695 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:20.915561 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 18:46:20.915695 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:20.915638 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 18:46:20.915695 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:20.915648 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-st4z8\"" Jul 09 18:46:20.928851 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:20.928822 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc"] Jul 09 18:46:21.050074 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.050038 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17c4b9fd-a28d-4960-aaa1-b336aa6497ec-tmp\") pod \"opendatahub-operator-controller-manager-fbd77dc5f-244nc\" (UID: \"17c4b9fd-a28d-4960-aaa1-b336aa6497ec\") " pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.050074 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.050075 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/17c4b9fd-a28d-4960-aaa1-b336aa6497ec-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fbd77dc5f-244nc\" (UID: \"17c4b9fd-a28d-4960-aaa1-b336aa6497ec\") " pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.050295 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.050106 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/17c4b9fd-a28d-4960-aaa1-b336aa6497ec-webhook-cert\") pod \"opendatahub-operator-controller-manager-fbd77dc5f-244nc\" (UID: \"17c4b9fd-a28d-4960-aaa1-b336aa6497ec\") " pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.050295 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.050199 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/17c4b9fd-a28d-4960-aaa1-b336aa6497ec-manifests\") pod \"opendatahub-operator-controller-manager-fbd77dc5f-244nc\" (UID: \"17c4b9fd-a28d-4960-aaa1-b336aa6497ec\") " pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.050360 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.050313 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwfmf\" (UniqueName: \"kubernetes.io/projected/17c4b9fd-a28d-4960-aaa1-b336aa6497ec-kube-api-access-fwfmf\") pod \"opendatahub-operator-controller-manager-fbd77dc5f-244nc\" (UID: \"17c4b9fd-a28d-4960-aaa1-b336aa6497ec\") " pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.151548 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.151512 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/17c4b9fd-a28d-4960-aaa1-b336aa6497ec-manifests\") pod \"opendatahub-operator-controller-manager-fbd77dc5f-244nc\" (UID: \"17c4b9fd-a28d-4960-aaa1-b336aa6497ec\") " pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.151737 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.151609 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwfmf\" (UniqueName: \"kubernetes.io/projected/17c4b9fd-a28d-4960-aaa1-b336aa6497ec-kube-api-access-fwfmf\") pod \"opendatahub-operator-controller-manager-fbd77dc5f-244nc\" (UID: \"17c4b9fd-a28d-4960-aaa1-b336aa6497ec\") " pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.151737 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.151675 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17c4b9fd-a28d-4960-aaa1-b336aa6497ec-tmp\") pod \"opendatahub-operator-controller-manager-fbd77dc5f-244nc\" (UID: \"17c4b9fd-a28d-4960-aaa1-b336aa6497ec\") " pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.151737 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.151698 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/17c4b9fd-a28d-4960-aaa1-b336aa6497ec-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fbd77dc5f-244nc\" (UID: \"17c4b9fd-a28d-4960-aaa1-b336aa6497ec\") " pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.151737 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.151727 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/17c4b9fd-a28d-4960-aaa1-b336aa6497ec-webhook-cert\") pod \"opendatahub-operator-controller-manager-fbd77dc5f-244nc\" (UID: \"17c4b9fd-a28d-4960-aaa1-b336aa6497ec\") " pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.152004 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.151977 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/17c4b9fd-a28d-4960-aaa1-b336aa6497ec-manifests\") pod \"opendatahub-operator-controller-manager-fbd77dc5f-244nc\" (UID: \"17c4b9fd-a28d-4960-aaa1-b336aa6497ec\") " pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.154257 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.154230 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17c4b9fd-a28d-4960-aaa1-b336aa6497ec-tmp\") pod \"opendatahub-operator-controller-manager-fbd77dc5f-244nc\" (UID: \"17c4b9fd-a28d-4960-aaa1-b336aa6497ec\") " pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.154518 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.154498 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/17c4b9fd-a28d-4960-aaa1-b336aa6497ec-webhook-cert\") pod \"opendatahub-operator-controller-manager-fbd77dc5f-244nc\" (UID: \"17c4b9fd-a28d-4960-aaa1-b336aa6497ec\") " pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.154828 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.154805 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/17c4b9fd-a28d-4960-aaa1-b336aa6497ec-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fbd77dc5f-244nc\" (UID: \"17c4b9fd-a28d-4960-aaa1-b336aa6497ec\") " pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.161090 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.161024 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwfmf\" (UniqueName: \"kubernetes.io/projected/17c4b9fd-a28d-4960-aaa1-b336aa6497ec-kube-api-access-fwfmf\") pod \"opendatahub-operator-controller-manager-fbd77dc5f-244nc\" (UID: \"17c4b9fd-a28d-4960-aaa1-b336aa6497ec\") " pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.223561 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.223523 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:21.381234 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:21.381196 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc"] Jul 09 18:46:21.385080 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:46:21.385048 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17c4b9fd_a28d_4960_aaa1_b336aa6497ec.slice/crio-68afbd18c0c42f6938aef60653a1e264b860bd9c0cc1b39f4698469b3c3b1f6b WatchSource:0}: Error finding container 68afbd18c0c42f6938aef60653a1e264b860bd9c0cc1b39f4698469b3c3b1f6b: Status 404 returned error can't find the container with id 68afbd18c0c42f6938aef60653a1e264b860bd9c0cc1b39f4698469b3c3b1f6b Jul 09 18:46:22.212988 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:22.212945 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" event={"ID":"17c4b9fd-a28d-4960-aaa1-b336aa6497ec","Type":"ContainerStarted","Data":"68afbd18c0c42f6938aef60653a1e264b860bd9c0cc1b39f4698469b3c3b1f6b"} Jul 09 18:46:24.366958 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:24.366933 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 18:46:24.366958 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:24.366947 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 18:46:24.373050 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:24.373028 2585 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 18:46:25.233539 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:25.233503 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" event={"ID":"17c4b9fd-a28d-4960-aaa1-b336aa6497ec","Type":"ContainerStarted","Data":"585c2cb0b2dda45145e3211045cca180699b9f1e5b70a18314fa2ae4636ecadb"} Jul 09 18:46:26.239113 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:26.239079 2585 generic.go:356] "Generic (PLEG): container finished" podID="17c4b9fd-a28d-4960-aaa1-b336aa6497ec" containerID="585c2cb0b2dda45145e3211045cca180699b9f1e5b70a18314fa2ae4636ecadb" exitCode=0 Jul 09 18:46:26.239512 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:26.239168 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" event={"ID":"17c4b9fd-a28d-4960-aaa1-b336aa6497ec","Type":"ContainerDied","Data":"585c2cb0b2dda45145e3211045cca180699b9f1e5b70a18314fa2ae4636ecadb"} Jul 09 18:46:26.239659 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:26.239645 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 18:46:27.245124 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:27.245086 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" event={"ID":"17c4b9fd-a28d-4960-aaa1-b336aa6497ec","Type":"ContainerStarted","Data":"a5cda29b357f84f5cf417998d15baf5f06f0fdbf251f8bb36558e9c7a55fbe3e"} Jul 09 18:46:27.245508 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:27.245220 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:27.285692 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:27.285643 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" podStartSLOduration=2.676228693 podStartE2EDuration="7.28562677s" totalImagesPullingTime="3.571681505s" totalInitContainerRuntime="1.037716572s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:46:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:46:27.282740941 +0000 UTC m=+304.026724055" watchObservedRunningTime="2026-07-09 18:46:27.28562677 +0000 UTC m=+304.029609883" Jul 09 18:46:38.250975 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:38.250944 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-fbd77dc5f-244nc" Jul 09 18:46:49.482333 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.482294 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5"] Jul 09 18:46:49.485521 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.485504 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:46:49.489762 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.489742 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 18:46:49.500740 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.500717 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 18:46:49.501196 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.501175 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8gfmv\"" Jul 09 18:46:49.501319 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.501178 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 18:46:49.503389 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.503364 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5"] Jul 09 18:46:49.581202 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.581165 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec-metrics-cert\") pod \"lws-controller-manager-68b945dcf-jnld5\" (UID: \"5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec\") " pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:46:49.581393 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.581238 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62gjt\" (UniqueName: \"kubernetes.io/projected/5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec-kube-api-access-62gjt\") pod \"lws-controller-manager-68b945dcf-jnld5\" (UID: \"5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec\") " pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:46:49.581393 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.581258 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec-manager-config\") pod \"lws-controller-manager-68b945dcf-jnld5\" (UID: \"5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec\") " pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:46:49.581393 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.581273 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec-cert\") pod \"lws-controller-manager-68b945dcf-jnld5\" (UID: \"5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec\") " pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:46:49.681731 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.681691 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec-metrics-cert\") pod \"lws-controller-manager-68b945dcf-jnld5\" (UID: \"5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec\") " pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:46:49.681904 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.681783 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62gjt\" (UniqueName: \"kubernetes.io/projected/5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec-kube-api-access-62gjt\") pod \"lws-controller-manager-68b945dcf-jnld5\" (UID: \"5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec\") " pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:46:49.681904 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.681814 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec-manager-config\") pod \"lws-controller-manager-68b945dcf-jnld5\" (UID: \"5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec\") " pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:46:49.681904 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.681834 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec-cert\") pod \"lws-controller-manager-68b945dcf-jnld5\" (UID: \"5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec\") " pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:46:49.682480 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.682460 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec-manager-config\") pod \"lws-controller-manager-68b945dcf-jnld5\" (UID: \"5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec\") " pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:46:49.684462 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.684430 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec-metrics-cert\") pod \"lws-controller-manager-68b945dcf-jnld5\" (UID: \"5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec\") " pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:46:49.684567 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.684493 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec-cert\") pod \"lws-controller-manager-68b945dcf-jnld5\" (UID: \"5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec\") " pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:46:49.693370 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.693346 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62gjt\" (UniqueName: \"kubernetes.io/projected/5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec-kube-api-access-62gjt\") pod \"lws-controller-manager-68b945dcf-jnld5\" (UID: \"5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec\") " pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:46:49.794663 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.794554 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:46:49.941048 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:49.940957 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5"] Jul 09 18:46:49.943319 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:46:49.943293 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5be5c13a_6e34_4dc1_ab0e_9ebe724f91ec.slice/crio-957ab6146e5afaaa543962ed507dfc7eb8bca2d9857de1a3a5491d6377c6cbae WatchSource:0}: Error finding container 957ab6146e5afaaa543962ed507dfc7eb8bca2d9857de1a3a5491d6377c6cbae: Status 404 returned error can't find the container with id 957ab6146e5afaaa543962ed507dfc7eb8bca2d9857de1a3a5491d6377c6cbae Jul 09 18:46:50.340948 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:50.340912 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" event={"ID":"5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec","Type":"ContainerStarted","Data":"957ab6146e5afaaa543962ed507dfc7eb8bca2d9857de1a3a5491d6377c6cbae"} Jul 09 18:46:51.689623 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.689560 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-74785d646-htfxm"] Jul 09 18:46:51.693316 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.693297 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74785d646-htfxm" Jul 09 18:46:51.696684 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.696659 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-t66rk\"" Jul 09 18:46:51.696818 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.696666 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 18:46:51.697785 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.697766 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 18:46:51.697863 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.697809 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 18:46:51.697923 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.697890 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 18:46:51.711303 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.711270 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74785d646-htfxm"] Jul 09 18:46:51.801990 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.801953 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fef65122-b577-4ccd-a8a6-5cebf06b7ab2-tmp\") pod \"kube-auth-proxy-74785d646-htfxm\" (UID: \"fef65122-b577-4ccd-a8a6-5cebf06b7ab2\") " pod="openshift-ingress/kube-auth-proxy-74785d646-htfxm" Jul 09 18:46:51.802168 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.802024 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fef65122-b577-4ccd-a8a6-5cebf06b7ab2-tls-certs\") pod \"kube-auth-proxy-74785d646-htfxm\" (UID: \"fef65122-b577-4ccd-a8a6-5cebf06b7ab2\") " pod="openshift-ingress/kube-auth-proxy-74785d646-htfxm" Jul 09 18:46:51.802168 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.802059 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgqz7\" (UniqueName: \"kubernetes.io/projected/fef65122-b577-4ccd-a8a6-5cebf06b7ab2-kube-api-access-bgqz7\") pod \"kube-auth-proxy-74785d646-htfxm\" (UID: \"fef65122-b577-4ccd-a8a6-5cebf06b7ab2\") " pod="openshift-ingress/kube-auth-proxy-74785d646-htfxm" Jul 09 18:46:51.902860 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.902816 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fef65122-b577-4ccd-a8a6-5cebf06b7ab2-tls-certs\") pod \"kube-auth-proxy-74785d646-htfxm\" (UID: \"fef65122-b577-4ccd-a8a6-5cebf06b7ab2\") " pod="openshift-ingress/kube-auth-proxy-74785d646-htfxm" Jul 09 18:46:51.902860 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.902859 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgqz7\" (UniqueName: \"kubernetes.io/projected/fef65122-b577-4ccd-a8a6-5cebf06b7ab2-kube-api-access-bgqz7\") pod \"kube-auth-proxy-74785d646-htfxm\" (UID: \"fef65122-b577-4ccd-a8a6-5cebf06b7ab2\") " pod="openshift-ingress/kube-auth-proxy-74785d646-htfxm" Jul 09 18:46:51.903122 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.902925 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fef65122-b577-4ccd-a8a6-5cebf06b7ab2-tmp\") pod \"kube-auth-proxy-74785d646-htfxm\" (UID: \"fef65122-b577-4ccd-a8a6-5cebf06b7ab2\") " pod="openshift-ingress/kube-auth-proxy-74785d646-htfxm" Jul 09 18:46:51.905369 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.905334 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fef65122-b577-4ccd-a8a6-5cebf06b7ab2-tmp\") pod \"kube-auth-proxy-74785d646-htfxm\" (UID: \"fef65122-b577-4ccd-a8a6-5cebf06b7ab2\") " pod="openshift-ingress/kube-auth-proxy-74785d646-htfxm" Jul 09 18:46:51.905481 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.905459 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fef65122-b577-4ccd-a8a6-5cebf06b7ab2-tls-certs\") pod \"kube-auth-proxy-74785d646-htfxm\" (UID: \"fef65122-b577-4ccd-a8a6-5cebf06b7ab2\") " pod="openshift-ingress/kube-auth-proxy-74785d646-htfxm" Jul 09 18:46:51.916919 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:51.916879 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgqz7\" (UniqueName: \"kubernetes.io/projected/fef65122-b577-4ccd-a8a6-5cebf06b7ab2-kube-api-access-bgqz7\") pod \"kube-auth-proxy-74785d646-htfxm\" (UID: \"fef65122-b577-4ccd-a8a6-5cebf06b7ab2\") " pod="openshift-ingress/kube-auth-proxy-74785d646-htfxm" Jul 09 18:46:52.005919 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:52.005832 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74785d646-htfxm" Jul 09 18:46:52.154556 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:52.154516 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74785d646-htfxm"] Jul 09 18:46:52.160325 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:46:52.159415 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfef65122_b577_4ccd_a8a6_5cebf06b7ab2.slice/crio-aa5a1751ca3caae99be5cbac1badf6c8994ba1e4c6dcdb3c6a5e4889eceb4c05 WatchSource:0}: Error finding container aa5a1751ca3caae99be5cbac1badf6c8994ba1e4c6dcdb3c6a5e4889eceb4c05: Status 404 returned error can't find the container with id aa5a1751ca3caae99be5cbac1badf6c8994ba1e4c6dcdb3c6a5e4889eceb4c05 Jul 09 18:46:52.351353 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:52.351264 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74785d646-htfxm" event={"ID":"fef65122-b577-4ccd-a8a6-5cebf06b7ab2","Type":"ContainerStarted","Data":"aa5a1751ca3caae99be5cbac1badf6c8994ba1e4c6dcdb3c6a5e4889eceb4c05"} Jul 09 18:46:56.376553 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:56.376516 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74785d646-htfxm" event={"ID":"fef65122-b577-4ccd-a8a6-5cebf06b7ab2","Type":"ContainerStarted","Data":"8dac77f8f5f6076787f4e36818091de948ee1d452e4bd728daeebf00ab11cee8"} Jul 09 18:46:56.401773 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:46:56.401726 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-74785d646-htfxm" podStartSLOduration=2.22038291 podStartE2EDuration="5.40171095s" totalImagesPullingTime="3.18132804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:46:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:46:56.399769376 +0000 UTC m=+333.143752502" watchObservedRunningTime="2026-07-09 18:46:56.40171095 +0000 UTC m=+333.145694064" Jul 09 18:47:15.460028 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:47:15.459988 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" event={"ID":"5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec","Type":"ContainerStarted","Data":"4108d647b4d9eaa9dba75c43559cf91497f3747e39f54dc2e4fd2a9796607ec8"} Jul 09 18:47:15.460459 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:47:15.460078 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:47:15.479952 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:47:15.479901 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" podStartSLOduration=1.876870816 podStartE2EDuration="26.479885475s" totalImagesPullingTime="24.603014659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:46:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:47:15.479012294 +0000 UTC m=+352.222995431" watchObservedRunningTime="2026-07-09 18:47:15.479885475 +0000 UTC m=+352.223868591" Jul 09 18:47:26.466400 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:47:26.466372 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-68b945dcf-jnld5" Jul 09 18:49:26.609339 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:26.609288 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sr6d7"] Jul 09 18:49:26.612762 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:26.612739 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" Jul 09 18:49:26.615361 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:26.615342 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 18:49:26.616410 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:26.616384 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 18:49:26.616524 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:26.616384 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9wdsc\"" Jul 09 18:49:26.616524 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:26.616459 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 18:49:26.624500 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:26.624477 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sr6d7"] Jul 09 18:49:26.699892 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:26.699863 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sr6d7"] Jul 09 18:49:26.794660 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:26.794628 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk29d\" (UniqueName: \"kubernetes.io/projected/5737c46f-ae43-4f44-8dab-18bec91edb5f-kube-api-access-rk29d\") pod \"limitador-limitador-64c94fd64f-sr6d7\" (UID: \"5737c46f-ae43-4f44-8dab-18bec91edb5f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" Jul 09 18:49:26.794819 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:26.794674 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5737c46f-ae43-4f44-8dab-18bec91edb5f-config-file\") pod \"limitador-limitador-64c94fd64f-sr6d7\" (UID: \"5737c46f-ae43-4f44-8dab-18bec91edb5f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" Jul 09 18:49:26.895656 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:26.895543 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk29d\" (UniqueName: \"kubernetes.io/projected/5737c46f-ae43-4f44-8dab-18bec91edb5f-kube-api-access-rk29d\") pod \"limitador-limitador-64c94fd64f-sr6d7\" (UID: \"5737c46f-ae43-4f44-8dab-18bec91edb5f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" Jul 09 18:49:26.895656 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:26.895603 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5737c46f-ae43-4f44-8dab-18bec91edb5f-config-file\") pod \"limitador-limitador-64c94fd64f-sr6d7\" (UID: \"5737c46f-ae43-4f44-8dab-18bec91edb5f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" Jul 09 18:49:26.896167 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:26.896149 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5737c46f-ae43-4f44-8dab-18bec91edb5f-config-file\") pod \"limitador-limitador-64c94fd64f-sr6d7\" (UID: \"5737c46f-ae43-4f44-8dab-18bec91edb5f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" Jul 09 18:49:26.906940 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:26.906910 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk29d\" (UniqueName: \"kubernetes.io/projected/5737c46f-ae43-4f44-8dab-18bec91edb5f-kube-api-access-rk29d\") pod \"limitador-limitador-64c94fd64f-sr6d7\" (UID: \"5737c46f-ae43-4f44-8dab-18bec91edb5f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" Jul 09 18:49:26.922661 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:26.922634 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" Jul 09 18:49:27.066624 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:27.066232 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sr6d7"] Jul 09 18:49:27.070510 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:49:27.070478 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5737c46f_ae43_4f44_8dab_18bec91edb5f.slice/crio-2d52b82e07f1b0dde741908c12de0990865798b7d3c22483b9a879993a0173a6 WatchSource:0}: Error finding container 2d52b82e07f1b0dde741908c12de0990865798b7d3c22483b9a879993a0173a6: Status 404 returned error can't find the container with id 2d52b82e07f1b0dde741908c12de0990865798b7d3c22483b9a879993a0173a6 Jul 09 18:49:27.830067 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:27.830030 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-45qh8"] Jul 09 18:49:27.833162 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:27.833144 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-45qh8" Jul 09 18:49:27.836379 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:27.836359 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xp8dl\"" Jul 09 18:49:27.842833 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:27.842812 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-45qh8"] Jul 09 18:49:28.003417 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:28.003289 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p7hv\" (UniqueName: \"kubernetes.io/projected/acdfdbfa-f0a9-4411-acf9-0081591350d5-kube-api-access-8p7hv\") pod \"authorino-75d556bd77-45qh8\" (UID: \"acdfdbfa-f0a9-4411-acf9-0081591350d5\") " pod="kuadrant-system/authorino-75d556bd77-45qh8" Jul 09 18:49:28.034133 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:28.034094 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" event={"ID":"5737c46f-ae43-4f44-8dab-18bec91edb5f","Type":"ContainerStarted","Data":"2d52b82e07f1b0dde741908c12de0990865798b7d3c22483b9a879993a0173a6"} Jul 09 18:49:28.104324 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:28.104231 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p7hv\" (UniqueName: \"kubernetes.io/projected/acdfdbfa-f0a9-4411-acf9-0081591350d5-kube-api-access-8p7hv\") pod \"authorino-75d556bd77-45qh8\" (UID: \"acdfdbfa-f0a9-4411-acf9-0081591350d5\") " pod="kuadrant-system/authorino-75d556bd77-45qh8" Jul 09 18:49:28.117457 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:28.117422 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p7hv\" (UniqueName: \"kubernetes.io/projected/acdfdbfa-f0a9-4411-acf9-0081591350d5-kube-api-access-8p7hv\") pod \"authorino-75d556bd77-45qh8\" (UID: \"acdfdbfa-f0a9-4411-acf9-0081591350d5\") " pod="kuadrant-system/authorino-75d556bd77-45qh8" Jul 09 18:49:28.144031 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:28.143999 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-45qh8" Jul 09 18:49:28.313150 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:28.313111 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-45qh8"] Jul 09 18:49:28.316409 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:49:28.316380 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podacdfdbfa_f0a9_4411_acf9_0081591350d5.slice/crio-a289c9364532e3f2ea3dd26750db6cd2ce2561d5653e9b8c7c5fb22767c81ca7 WatchSource:0}: Error finding container a289c9364532e3f2ea3dd26750db6cd2ce2561d5653e9b8c7c5fb22767c81ca7: Status 404 returned error can't find the container with id a289c9364532e3f2ea3dd26750db6cd2ce2561d5653e9b8c7c5fb22767c81ca7 Jul 09 18:49:29.040536 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:29.040483 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-45qh8" event={"ID":"acdfdbfa-f0a9-4411-acf9-0081591350d5","Type":"ContainerStarted","Data":"a289c9364532e3f2ea3dd26750db6cd2ce2561d5653e9b8c7c5fb22767c81ca7"} Jul 09 18:49:31.053828 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:31.053788 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" event={"ID":"5737c46f-ae43-4f44-8dab-18bec91edb5f","Type":"ContainerStarted","Data":"6ad5f079ff2a564bf85ef8d51b3665bcb3f398973bdbf93d56e31dfb2156b50f"} Jul 09 18:49:31.054200 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:31.053914 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" Jul 09 18:49:31.072653 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:31.072603 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" podStartSLOduration=2.102465194 podStartE2EDuration="5.072561753s" totalImagesPullingTime="2.970096559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:49:31.072096215 +0000 UTC m=+487.816079370" watchObservedRunningTime="2026-07-09 18:49:31.072561753 +0000 UTC m=+487.816544868" Jul 09 18:49:32.060914 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:32.060871 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-45qh8" event={"ID":"acdfdbfa-f0a9-4411-acf9-0081591350d5","Type":"ContainerStarted","Data":"c22dbb4a0481eaab38aa0e13c2b7ff0267a3b86478ac10981d3270d68a02e19d"} Jul 09 18:49:32.080979 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:32.080911 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-45qh8" podStartSLOduration=1.465520747 podStartE2EDuration="5.080889352s" totalImagesPullingTime="3.615368605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:49:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:49:32.078767726 +0000 UTC m=+488.822750841" watchObservedRunningTime="2026-07-09 18:49:32.080889352 +0000 UTC m=+488.824872467" Jul 09 18:49:42.062529 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:42.062498 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" Jul 09 18:49:44.031165 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:44.031123 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sr6d7"] Jul 09 18:49:44.031619 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:44.031314 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" podUID="5737c46f-ae43-4f44-8dab-18bec91edb5f" containerName="limitador" containerID="cri-o://6ad5f079ff2a564bf85ef8d51b3665bcb3f398973bdbf93d56e31dfb2156b50f" gracePeriod=30 Jul 09 18:49:44.584133 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:44.584108 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" Jul 09 18:49:44.643627 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:44.643510 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5737c46f-ae43-4f44-8dab-18bec91edb5f-config-file\") pod \"5737c46f-ae43-4f44-8dab-18bec91edb5f\" (UID: \"5737c46f-ae43-4f44-8dab-18bec91edb5f\") " Jul 09 18:49:44.643796 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:44.643650 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rk29d\" (UniqueName: \"kubernetes.io/projected/5737c46f-ae43-4f44-8dab-18bec91edb5f-kube-api-access-rk29d\") pod \"5737c46f-ae43-4f44-8dab-18bec91edb5f\" (UID: \"5737c46f-ae43-4f44-8dab-18bec91edb5f\") " Jul 09 18:49:44.643886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:44.643811 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5737c46f-ae43-4f44-8dab-18bec91edb5f-config-file" (OuterVolumeSpecName: "config-file") pod "5737c46f-ae43-4f44-8dab-18bec91edb5f" (UID: "5737c46f-ae43-4f44-8dab-18bec91edb5f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:49:44.643945 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:44.643934 2585 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5737c46f-ae43-4f44-8dab-18bec91edb5f-config-file\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:49:44.646149 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:44.646122 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5737c46f-ae43-4f44-8dab-18bec91edb5f-kube-api-access-rk29d" (OuterVolumeSpecName: "kube-api-access-rk29d") pod "5737c46f-ae43-4f44-8dab-18bec91edb5f" (UID: "5737c46f-ae43-4f44-8dab-18bec91edb5f"). InnerVolumeSpecName "kube-api-access-rk29d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:49:44.744663 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:44.744629 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rk29d\" (UniqueName: \"kubernetes.io/projected/5737c46f-ae43-4f44-8dab-18bec91edb5f-kube-api-access-rk29d\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:49:45.117426 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:45.117394 2585 generic.go:356] "Generic (PLEG): container finished" podID="5737c46f-ae43-4f44-8dab-18bec91edb5f" containerID="6ad5f079ff2a564bf85ef8d51b3665bcb3f398973bdbf93d56e31dfb2156b50f" exitCode=0 Jul 09 18:49:45.117951 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:45.117453 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" Jul 09 18:49:45.117951 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:45.117491 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" event={"ID":"5737c46f-ae43-4f44-8dab-18bec91edb5f","Type":"ContainerDied","Data":"6ad5f079ff2a564bf85ef8d51b3665bcb3f398973bdbf93d56e31dfb2156b50f"} Jul 09 18:49:45.117951 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:45.117562 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sr6d7" event={"ID":"5737c46f-ae43-4f44-8dab-18bec91edb5f","Type":"ContainerDied","Data":"2d52b82e07f1b0dde741908c12de0990865798b7d3c22483b9a879993a0173a6"} Jul 09 18:49:45.117951 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:45.117627 2585 scope.go:117] "RemoveContainer" containerID="6ad5f079ff2a564bf85ef8d51b3665bcb3f398973bdbf93d56e31dfb2156b50f" Jul 09 18:49:45.126860 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:45.126843 2585 scope.go:117] "RemoveContainer" containerID="6ad5f079ff2a564bf85ef8d51b3665bcb3f398973bdbf93d56e31dfb2156b50f" Jul 09 18:49:45.127118 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:49:45.127099 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ad5f079ff2a564bf85ef8d51b3665bcb3f398973bdbf93d56e31dfb2156b50f\": container with ID starting with 6ad5f079ff2a564bf85ef8d51b3665bcb3f398973bdbf93d56e31dfb2156b50f not found: ID does not exist" containerID="6ad5f079ff2a564bf85ef8d51b3665bcb3f398973bdbf93d56e31dfb2156b50f" Jul 09 18:49:45.127195 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:45.127127 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ad5f079ff2a564bf85ef8d51b3665bcb3f398973bdbf93d56e31dfb2156b50f"} err="failed to get container status \"6ad5f079ff2a564bf85ef8d51b3665bcb3f398973bdbf93d56e31dfb2156b50f\": rpc error: code = NotFound desc = could not find container \"6ad5f079ff2a564bf85ef8d51b3665bcb3f398973bdbf93d56e31dfb2156b50f\": container with ID starting with 6ad5f079ff2a564bf85ef8d51b3665bcb3f398973bdbf93d56e31dfb2156b50f not found: ID does not exist" Jul 09 18:49:45.148895 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:45.148861 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sr6d7"] Jul 09 18:49:45.156984 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:45.156957 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sr6d7"] Jul 09 18:49:45.843130 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:45.843087 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5737c46f-ae43-4f44-8dab-18bec91edb5f" path="/var/lib/kubelet/pods/5737c46f-ae43-4f44-8dab-18bec91edb5f/volumes" Jul 09 18:49:46.898507 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:46.898470 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q"] Jul 09 18:49:46.898902 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:46.898825 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5737c46f-ae43-4f44-8dab-18bec91edb5f" containerName="limitador" Jul 09 18:49:46.898902 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:46.898836 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5737c46f-ae43-4f44-8dab-18bec91edb5f" containerName="limitador" Jul 09 18:49:46.898902 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:46.898898 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5737c46f-ae43-4f44-8dab-18bec91edb5f" containerName="limitador" Jul 09 18:49:46.901893 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:46.901877 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q" Jul 09 18:49:46.905135 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:46.905104 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 18:49:46.905135 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:46.905123 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 09 18:49:46.905292 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:46.905195 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 18:49:46.906443 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:46.906429 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-gfhn5\"" Jul 09 18:49:46.918063 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:46.918041 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q"] Jul 09 18:49:46.962267 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:46.962235 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqd9f\" (UniqueName: \"kubernetes.io/projected/d3ca36a1-0fae-4899-adc5-73b5d165b972-kube-api-access-mqd9f\") pod \"postgres-69fb4594cf-r5f7q\" (UID: \"d3ca36a1-0fae-4899-adc5-73b5d165b972\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q" Jul 09 18:49:46.962381 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:46.962281 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d3ca36a1-0fae-4899-adc5-73b5d165b972-data\") pod \"postgres-69fb4594cf-r5f7q\" (UID: \"d3ca36a1-0fae-4899-adc5-73b5d165b972\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q" Jul 09 18:49:47.063135 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:47.063101 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqd9f\" (UniqueName: \"kubernetes.io/projected/d3ca36a1-0fae-4899-adc5-73b5d165b972-kube-api-access-mqd9f\") pod \"postgres-69fb4594cf-r5f7q\" (UID: \"d3ca36a1-0fae-4899-adc5-73b5d165b972\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q" Jul 09 18:49:47.063305 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:47.063150 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d3ca36a1-0fae-4899-adc5-73b5d165b972-data\") pod \"postgres-69fb4594cf-r5f7q\" (UID: \"d3ca36a1-0fae-4899-adc5-73b5d165b972\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q" Jul 09 18:49:47.063519 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:47.063502 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d3ca36a1-0fae-4899-adc5-73b5d165b972-data\") pod \"postgres-69fb4594cf-r5f7q\" (UID: \"d3ca36a1-0fae-4899-adc5-73b5d165b972\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q" Jul 09 18:49:47.129032 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:47.129002 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqd9f\" (UniqueName: \"kubernetes.io/projected/d3ca36a1-0fae-4899-adc5-73b5d165b972-kube-api-access-mqd9f\") pod \"postgres-69fb4594cf-r5f7q\" (UID: \"d3ca36a1-0fae-4899-adc5-73b5d165b972\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q" Jul 09 18:49:47.211958 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:47.211927 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q" Jul 09 18:49:47.406185 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:47.406148 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q"] Jul 09 18:49:47.432537 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:49:47.432512 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3ca36a1_0fae_4899_adc5_73b5d165b972.slice/crio-ad471db62c7ae46cff1aa1247f0b9118eb0787f8c3f14df941a8e53731d4d8b2 WatchSource:0}: Error finding container ad471db62c7ae46cff1aa1247f0b9118eb0787f8c3f14df941a8e53731d4d8b2: Status 404 returned error can't find the container with id ad471db62c7ae46cff1aa1247f0b9118eb0787f8c3f14df941a8e53731d4d8b2 Jul 09 18:49:48.132346 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:48.132316 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q" event={"ID":"d3ca36a1-0fae-4899-adc5-73b5d165b972","Type":"ContainerStarted","Data":"ad471db62c7ae46cff1aa1247f0b9118eb0787f8c3f14df941a8e53731d4d8b2"} Jul 09 18:49:53.160083 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:53.160042 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q" event={"ID":"d3ca36a1-0fae-4899-adc5-73b5d165b972","Type":"ContainerStarted","Data":"81c11b0191453de7edee4883763c91a447c7aa09cbf26ffdafa21ea57fd83968"} Jul 09 18:49:53.160483 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:53.160170 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q" Jul 09 18:49:53.179179 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:53.179129 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q" podStartSLOduration=2.158824091 podStartE2EDuration="7.179114426s" totalImagesPullingTime="5.020290335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:49:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:49:53.177418629 +0000 UTC m=+509.921401742" watchObservedRunningTime="2026-07-09 18:49:53.179114426 +0000 UTC m=+509.923097540" Jul 09 18:49:59.194452 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:49:59.194422 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-r5f7q" Jul 09 18:50:01.156666 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:01.156630 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-45qh8"] Jul 09 18:50:01.157041 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:01.156821 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-45qh8" podUID="acdfdbfa-f0a9-4411-acf9-0081591350d5" containerName="authorino" containerID="cri-o://c22dbb4a0481eaab38aa0e13c2b7ff0267a3b86478ac10981d3270d68a02e19d" gracePeriod=30 Jul 09 18:50:01.404116 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:01.404092 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-45qh8" Jul 09 18:50:01.491606 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:01.491548 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p7hv\" (UniqueName: \"kubernetes.io/projected/acdfdbfa-f0a9-4411-acf9-0081591350d5-kube-api-access-8p7hv\") pod \"acdfdbfa-f0a9-4411-acf9-0081591350d5\" (UID: \"acdfdbfa-f0a9-4411-acf9-0081591350d5\") " Jul 09 18:50:01.493826 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:01.493794 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acdfdbfa-f0a9-4411-acf9-0081591350d5-kube-api-access-8p7hv" (OuterVolumeSpecName: "kube-api-access-8p7hv") pod "acdfdbfa-f0a9-4411-acf9-0081591350d5" (UID: "acdfdbfa-f0a9-4411-acf9-0081591350d5"). InnerVolumeSpecName "kube-api-access-8p7hv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:50:01.592559 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:01.592527 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p7hv\" (UniqueName: \"kubernetes.io/projected/acdfdbfa-f0a9-4411-acf9-0081591350d5-kube-api-access-8p7hv\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:50:02.201536 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:02.201507 2585 generic.go:356] "Generic (PLEG): container finished" podID="acdfdbfa-f0a9-4411-acf9-0081591350d5" containerID="c22dbb4a0481eaab38aa0e13c2b7ff0267a3b86478ac10981d3270d68a02e19d" exitCode=0 Jul 09 18:50:02.201958 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:02.201551 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-45qh8" Jul 09 18:50:02.201958 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:02.201612 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-45qh8" event={"ID":"acdfdbfa-f0a9-4411-acf9-0081591350d5","Type":"ContainerDied","Data":"c22dbb4a0481eaab38aa0e13c2b7ff0267a3b86478ac10981d3270d68a02e19d"} Jul 09 18:50:02.201958 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:02.201654 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-45qh8" event={"ID":"acdfdbfa-f0a9-4411-acf9-0081591350d5","Type":"ContainerDied","Data":"a289c9364532e3f2ea3dd26750db6cd2ce2561d5653e9b8c7c5fb22767c81ca7"} Jul 09 18:50:02.201958 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:02.201671 2585 scope.go:117] "RemoveContainer" containerID="c22dbb4a0481eaab38aa0e13c2b7ff0267a3b86478ac10981d3270d68a02e19d" Jul 09 18:50:02.210025 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:02.210009 2585 scope.go:117] "RemoveContainer" containerID="c22dbb4a0481eaab38aa0e13c2b7ff0267a3b86478ac10981d3270d68a02e19d" Jul 09 18:50:02.210274 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:50:02.210255 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c22dbb4a0481eaab38aa0e13c2b7ff0267a3b86478ac10981d3270d68a02e19d\": container with ID starting with c22dbb4a0481eaab38aa0e13c2b7ff0267a3b86478ac10981d3270d68a02e19d not found: ID does not exist" containerID="c22dbb4a0481eaab38aa0e13c2b7ff0267a3b86478ac10981d3270d68a02e19d" Jul 09 18:50:02.210317 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:02.210282 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c22dbb4a0481eaab38aa0e13c2b7ff0267a3b86478ac10981d3270d68a02e19d"} err="failed to get container status \"c22dbb4a0481eaab38aa0e13c2b7ff0267a3b86478ac10981d3270d68a02e19d\": rpc error: code = NotFound desc = could not find container \"c22dbb4a0481eaab38aa0e13c2b7ff0267a3b86478ac10981d3270d68a02e19d\": container with ID starting with c22dbb4a0481eaab38aa0e13c2b7ff0267a3b86478ac10981d3270d68a02e19d not found: ID does not exist" Jul 09 18:50:02.221559 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:02.221537 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-45qh8"] Jul 09 18:50:02.223576 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:02.223555 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-45qh8"] Jul 09 18:50:03.843450 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:03.843374 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acdfdbfa-f0a9-4411-acf9-0081591350d5" path="/var/lib/kubelet/pods/acdfdbfa-f0a9-4411-acf9-0081591350d5/volumes" Jul 09 18:50:05.710435 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.710400 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-674c5757fd-4kwxs"] Jul 09 18:50:05.710843 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.710747 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acdfdbfa-f0a9-4411-acf9-0081591350d5" containerName="authorino" Jul 09 18:50:05.710843 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.710759 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="acdfdbfa-f0a9-4411-acf9-0081591350d5" containerName="authorino" Jul 09 18:50:05.710843 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.710823 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="acdfdbfa-f0a9-4411-acf9-0081591350d5" containerName="authorino" Jul 09 18:50:05.715180 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.715164 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-674c5757fd-4kwxs" Jul 09 18:50:05.718738 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.718710 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-gmrv2\"" Jul 09 18:50:05.718878 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.718860 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 18:50:05.719575 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.719560 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 18:50:05.730308 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.730283 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-674c5757fd-4kwxs"] Jul 09 18:50:05.829510 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.829475 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/577a990d-e8f5-4995-a3a6-7b4a0879db71-service-ca\") pod \"maas-controller-674c5757fd-4kwxs\" (UID: \"577a990d-e8f5-4995-a3a6-7b4a0879db71\") " pod="opendatahub/maas-controller-674c5757fd-4kwxs" Jul 09 18:50:05.829510 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.829520 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mttcr\" (UniqueName: \"kubernetes.io/projected/577a990d-e8f5-4995-a3a6-7b4a0879db71-kube-api-access-mttcr\") pod \"maas-controller-674c5757fd-4kwxs\" (UID: \"577a990d-e8f5-4995-a3a6-7b4a0879db71\") " pod="opendatahub/maas-controller-674c5757fd-4kwxs" Jul 09 18:50:05.829752 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.829653 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/577a990d-e8f5-4995-a3a6-7b4a0879db71-webhook-cert\") pod \"maas-controller-674c5757fd-4kwxs\" (UID: \"577a990d-e8f5-4995-a3a6-7b4a0879db71\") " pod="opendatahub/maas-controller-674c5757fd-4kwxs" Jul 09 18:50:05.921615 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.921557 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f5cd9bc7b-7cwpd"] Jul 09 18:50:05.924800 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.924783 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" Jul 09 18:50:05.930188 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.930161 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/577a990d-e8f5-4995-a3a6-7b4a0879db71-service-ca\") pod \"maas-controller-674c5757fd-4kwxs\" (UID: \"577a990d-e8f5-4995-a3a6-7b4a0879db71\") " pod="opendatahub/maas-controller-674c5757fd-4kwxs" Jul 09 18:50:05.930343 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.930210 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mttcr\" (UniqueName: \"kubernetes.io/projected/577a990d-e8f5-4995-a3a6-7b4a0879db71-kube-api-access-mttcr\") pod \"maas-controller-674c5757fd-4kwxs\" (UID: \"577a990d-e8f5-4995-a3a6-7b4a0879db71\") " pod="opendatahub/maas-controller-674c5757fd-4kwxs" Jul 09 18:50:05.930414 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.930373 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/577a990d-e8f5-4995-a3a6-7b4a0879db71-webhook-cert\") pod \"maas-controller-674c5757fd-4kwxs\" (UID: \"577a990d-e8f5-4995-a3a6-7b4a0879db71\") " pod="opendatahub/maas-controller-674c5757fd-4kwxs" Jul 09 18:50:05.931008 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.930987 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/577a990d-e8f5-4995-a3a6-7b4a0879db71-service-ca\") pod \"maas-controller-674c5757fd-4kwxs\" (UID: \"577a990d-e8f5-4995-a3a6-7b4a0879db71\") " pod="opendatahub/maas-controller-674c5757fd-4kwxs" Jul 09 18:50:05.933085 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.933063 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/577a990d-e8f5-4995-a3a6-7b4a0879db71-webhook-cert\") pod \"maas-controller-674c5757fd-4kwxs\" (UID: \"577a990d-e8f5-4995-a3a6-7b4a0879db71\") " pod="opendatahub/maas-controller-674c5757fd-4kwxs" Jul 09 18:50:05.946325 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.946297 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mttcr\" (UniqueName: \"kubernetes.io/projected/577a990d-e8f5-4995-a3a6-7b4a0879db71-kube-api-access-mttcr\") pod \"maas-controller-674c5757fd-4kwxs\" (UID: \"577a990d-e8f5-4995-a3a6-7b4a0879db71\") " pod="opendatahub/maas-controller-674c5757fd-4kwxs" Jul 09 18:50:05.946622 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:05.946602 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f5cd9bc7b-7cwpd"] Jul 09 18:50:06.025808 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:06.025695 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-674c5757fd-4kwxs" Jul 09 18:50:06.031071 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:06.031047 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-webhook-cert\") pod \"maas-controller-5f5cd9bc7b-7cwpd\" (UID: \"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041\") " pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" Jul 09 18:50:06.031174 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:06.031085 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-service-ca\") pod \"maas-controller-5f5cd9bc7b-7cwpd\" (UID: \"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041\") " pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" Jul 09 18:50:06.031174 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:06.031119 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t76mj\" (UniqueName: \"kubernetes.io/projected/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-kube-api-access-t76mj\") pod \"maas-controller-5f5cd9bc7b-7cwpd\" (UID: \"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041\") " pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" Jul 09 18:50:06.132190 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:06.132161 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-webhook-cert\") pod \"maas-controller-5f5cd9bc7b-7cwpd\" (UID: \"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041\") " pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" Jul 09 18:50:06.132341 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:06.132196 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-service-ca\") pod \"maas-controller-5f5cd9bc7b-7cwpd\" (UID: \"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041\") " pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" Jul 09 18:50:06.132341 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:06.132227 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t76mj\" (UniqueName: \"kubernetes.io/projected/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-kube-api-access-t76mj\") pod \"maas-controller-5f5cd9bc7b-7cwpd\" (UID: \"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041\") " pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" Jul 09 18:50:06.133120 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:06.133092 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-service-ca\") pod \"maas-controller-5f5cd9bc7b-7cwpd\" (UID: \"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041\") " pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" Jul 09 18:50:06.135038 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:06.135018 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-webhook-cert\") pod \"maas-controller-5f5cd9bc7b-7cwpd\" (UID: \"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041\") " pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" Jul 09 18:50:06.145094 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:06.145065 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t76mj\" (UniqueName: \"kubernetes.io/projected/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-kube-api-access-t76mj\") pod \"maas-controller-5f5cd9bc7b-7cwpd\" (UID: \"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041\") " pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" Jul 09 18:50:06.191893 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:06.191738 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-674c5757fd-4kwxs"] Jul 09 18:50:06.194739 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:50:06.194712 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod577a990d_e8f5_4995_a3a6_7b4a0879db71.slice/crio-9ec192eded45f83b47f8ada148b3253a36d661038ac864d58e7e38686f71b7a2 WatchSource:0}: Error finding container 9ec192eded45f83b47f8ada148b3253a36d661038ac864d58e7e38686f71b7a2: Status 404 returned error can't find the container with id 9ec192eded45f83b47f8ada148b3253a36d661038ac864d58e7e38686f71b7a2 Jul 09 18:50:06.221388 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:06.221357 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-674c5757fd-4kwxs" event={"ID":"577a990d-e8f5-4995-a3a6-7b4a0879db71","Type":"ContainerStarted","Data":"9ec192eded45f83b47f8ada148b3253a36d661038ac864d58e7e38686f71b7a2"} Jul 09 18:50:06.234741 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:06.234701 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" Jul 09 18:50:06.379187 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:06.379159 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f5cd9bc7b-7cwpd"] Jul 09 18:50:06.381822 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:50:06.381788 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c8dbb5_a8b8_4cf8_aa0f_0b388daae041.slice/crio-f5eb578209a37b1b58250be666a3c97ba2943465e98d4bf975451b47cc7d2d5b WatchSource:0}: Error finding container f5eb578209a37b1b58250be666a3c97ba2943465e98d4bf975451b47cc7d2d5b: Status 404 returned error can't find the container with id f5eb578209a37b1b58250be666a3c97ba2943465e98d4bf975451b47cc7d2d5b Jul 09 18:50:07.228107 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:07.228066 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" event={"ID":"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041","Type":"ContainerStarted","Data":"f5eb578209a37b1b58250be666a3c97ba2943465e98d4bf975451b47cc7d2d5b"} Jul 09 18:50:09.239518 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:09.239479 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-674c5757fd-4kwxs" event={"ID":"577a990d-e8f5-4995-a3a6-7b4a0879db71","Type":"ContainerStarted","Data":"cb72b16b75439470b5290bdccff034812778ac87b4f9e8c97dd60b561c20df0f"} Jul 09 18:50:09.239970 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:09.239553 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-674c5757fd-4kwxs" Jul 09 18:50:09.241078 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:09.241051 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" event={"ID":"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041","Type":"ContainerStarted","Data":"554afc719c1a57dd6d6ab11f516bb790d4cf6b413e45d212f4aafa116706dadf"} Jul 09 18:50:09.241190 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:09.241175 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" Jul 09 18:50:09.289883 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:09.289832 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-674c5757fd-4kwxs" podStartSLOduration=1.791476699 podStartE2EDuration="4.289815098s" totalImagesPullingTime="2.498338399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:50:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:50:09.285653995 +0000 UTC m=+526.029637100" watchObservedRunningTime="2026-07-09 18:50:09.289815098 +0000 UTC m=+526.033798212" Jul 09 18:50:09.326055 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:09.326001 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" podStartSLOduration=2.025849751 podStartE2EDuration="4.325985382s" totalImagesPullingTime="2.300135631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:50:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:50:09.323830453 +0000 UTC m=+526.067813572" watchObservedRunningTime="2026-07-09 18:50:09.325985382 +0000 UTC m=+526.069968496" Jul 09 18:50:20.251399 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:20.251323 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" Jul 09 18:50:20.251931 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:20.251641 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-674c5757fd-4kwxs" Jul 09 18:50:20.327060 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:20.327024 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-674c5757fd-4kwxs"] Jul 09 18:50:20.327297 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:20.327248 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-674c5757fd-4kwxs" podUID="577a990d-e8f5-4995-a3a6-7b4a0879db71" containerName="manager" containerID="cri-o://cb72b16b75439470b5290bdccff034812778ac87b4f9e8c97dd60b561c20df0f" gracePeriod=30 Jul 09 18:50:20.569906 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:20.569881 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-674c5757fd-4kwxs" Jul 09 18:50:20.664248 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:20.664219 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mttcr\" (UniqueName: \"kubernetes.io/projected/577a990d-e8f5-4995-a3a6-7b4a0879db71-kube-api-access-mttcr\") pod \"577a990d-e8f5-4995-a3a6-7b4a0879db71\" (UID: \"577a990d-e8f5-4995-a3a6-7b4a0879db71\") " Jul 09 18:50:20.664444 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:20.664264 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/577a990d-e8f5-4995-a3a6-7b4a0879db71-webhook-cert\") pod \"577a990d-e8f5-4995-a3a6-7b4a0879db71\" (UID: \"577a990d-e8f5-4995-a3a6-7b4a0879db71\") " Jul 09 18:50:20.664444 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:20.664325 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/577a990d-e8f5-4995-a3a6-7b4a0879db71-service-ca\") pod \"577a990d-e8f5-4995-a3a6-7b4a0879db71\" (UID: \"577a990d-e8f5-4995-a3a6-7b4a0879db71\") " Jul 09 18:50:20.664730 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:20.664703 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/577a990d-e8f5-4995-a3a6-7b4a0879db71-service-ca" (OuterVolumeSpecName: "service-ca") pod "577a990d-e8f5-4995-a3a6-7b4a0879db71" (UID: "577a990d-e8f5-4995-a3a6-7b4a0879db71"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:50:20.666685 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:20.666646 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577a990d-e8f5-4995-a3a6-7b4a0879db71-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "577a990d-e8f5-4995-a3a6-7b4a0879db71" (UID: "577a990d-e8f5-4995-a3a6-7b4a0879db71"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:50:20.666795 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:20.666739 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/577a990d-e8f5-4995-a3a6-7b4a0879db71-kube-api-access-mttcr" (OuterVolumeSpecName: "kube-api-access-mttcr") pod "577a990d-e8f5-4995-a3a6-7b4a0879db71" (UID: "577a990d-e8f5-4995-a3a6-7b4a0879db71"). InnerVolumeSpecName "kube-api-access-mttcr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:50:20.764933 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:20.764898 2585 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/577a990d-e8f5-4995-a3a6-7b4a0879db71-webhook-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:50:20.764933 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:20.764927 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/577a990d-e8f5-4995-a3a6-7b4a0879db71-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:50:20.764933 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:20.764939 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mttcr\" (UniqueName: \"kubernetes.io/projected/577a990d-e8f5-4995-a3a6-7b4a0879db71-kube-api-access-mttcr\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:50:21.300519 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:21.300487 2585 generic.go:356] "Generic (PLEG): container finished" podID="577a990d-e8f5-4995-a3a6-7b4a0879db71" containerID="cb72b16b75439470b5290bdccff034812778ac87b4f9e8c97dd60b561c20df0f" exitCode=0 Jul 09 18:50:21.300519 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:21.300524 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-674c5757fd-4kwxs" event={"ID":"577a990d-e8f5-4995-a3a6-7b4a0879db71","Type":"ContainerDied","Data":"cb72b16b75439470b5290bdccff034812778ac87b4f9e8c97dd60b561c20df0f"} Jul 09 18:50:21.301006 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:21.300529 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-674c5757fd-4kwxs" Jul 09 18:50:21.301006 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:21.300571 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-674c5757fd-4kwxs" event={"ID":"577a990d-e8f5-4995-a3a6-7b4a0879db71","Type":"ContainerDied","Data":"9ec192eded45f83b47f8ada148b3253a36d661038ac864d58e7e38686f71b7a2"} Jul 09 18:50:21.301006 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:21.300611 2585 scope.go:117] "RemoveContainer" containerID="cb72b16b75439470b5290bdccff034812778ac87b4f9e8c97dd60b561c20df0f" Jul 09 18:50:21.309556 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:21.309538 2585 scope.go:117] "RemoveContainer" containerID="cb72b16b75439470b5290bdccff034812778ac87b4f9e8c97dd60b561c20df0f" Jul 09 18:50:21.309841 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:50:21.309823 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb72b16b75439470b5290bdccff034812778ac87b4f9e8c97dd60b561c20df0f\": container with ID starting with cb72b16b75439470b5290bdccff034812778ac87b4f9e8c97dd60b561c20df0f not found: ID does not exist" containerID="cb72b16b75439470b5290bdccff034812778ac87b4f9e8c97dd60b561c20df0f" Jul 09 18:50:21.309912 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:21.309851 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb72b16b75439470b5290bdccff034812778ac87b4f9e8c97dd60b561c20df0f"} err="failed to get container status \"cb72b16b75439470b5290bdccff034812778ac87b4f9e8c97dd60b561c20df0f\": rpc error: code = NotFound desc = could not find container \"cb72b16b75439470b5290bdccff034812778ac87b4f9e8c97dd60b561c20df0f\": container with ID starting with cb72b16b75439470b5290bdccff034812778ac87b4f9e8c97dd60b561c20df0f not found: ID does not exist" Jul 09 18:50:21.326489 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:21.326463 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-674c5757fd-4kwxs"] Jul 09 18:50:21.329253 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:21.329232 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-674c5757fd-4kwxs"] Jul 09 18:50:21.842509 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:21.842443 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="577a990d-e8f5-4995-a3a6-7b4a0879db71" path="/var/lib/kubelet/pods/577a990d-e8f5-4995-a3a6-7b4a0879db71/volumes" Jul 09 18:50:40.872387 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:40.872353 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk"] Jul 09 18:50:40.872895 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:40.872696 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="577a990d-e8f5-4995-a3a6-7b4a0879db71" containerName="manager" Jul 09 18:50:40.872895 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:40.872707 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="577a990d-e8f5-4995-a3a6-7b4a0879db71" containerName="manager" Jul 09 18:50:40.872895 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:40.872770 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="577a990d-e8f5-4995-a3a6-7b4a0879db71" containerName="manager" Jul 09 18:50:40.900675 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:40.900641 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk"] Jul 09 18:50:40.900823 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:40.900740 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" Jul 09 18:50:40.903267 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:40.903247 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 09 18:50:40.903484 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:40.903472 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jq94v\"" Jul 09 18:50:40.933185 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:40.933159 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/235c900d-b372-41f9-bf33-0f1a5bf90e63-openshift-service-ca\") pod \"maas-api-d89b88c4c-ccnjk\" (UID: \"235c900d-b372-41f9-bf33-0f1a5bf90e63\") " pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" Jul 09 18:50:40.933335 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:40.933315 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235c900d-b372-41f9-bf33-0f1a5bf90e63-maas-api-tls\") pod \"maas-api-d89b88c4c-ccnjk\" (UID: \"235c900d-b372-41f9-bf33-0f1a5bf90e63\") " pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" Jul 09 18:50:40.933399 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:40.933347 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8fz5\" (UniqueName: \"kubernetes.io/projected/235c900d-b372-41f9-bf33-0f1a5bf90e63-kube-api-access-k8fz5\") pod \"maas-api-d89b88c4c-ccnjk\" (UID: \"235c900d-b372-41f9-bf33-0f1a5bf90e63\") " pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" Jul 09 18:50:41.034359 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.034329 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235c900d-b372-41f9-bf33-0f1a5bf90e63-maas-api-tls\") pod \"maas-api-d89b88c4c-ccnjk\" (UID: \"235c900d-b372-41f9-bf33-0f1a5bf90e63\") " pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" Jul 09 18:50:41.034359 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.034363 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8fz5\" (UniqueName: \"kubernetes.io/projected/235c900d-b372-41f9-bf33-0f1a5bf90e63-kube-api-access-k8fz5\") pod \"maas-api-d89b88c4c-ccnjk\" (UID: \"235c900d-b372-41f9-bf33-0f1a5bf90e63\") " pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" Jul 09 18:50:41.034555 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.034401 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/235c900d-b372-41f9-bf33-0f1a5bf90e63-openshift-service-ca\") pod \"maas-api-d89b88c4c-ccnjk\" (UID: \"235c900d-b372-41f9-bf33-0f1a5bf90e63\") " pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" Jul 09 18:50:41.034555 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:50:41.034470 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 09 18:50:41.034555 ip-10-0-143-175 kubenswrapper[2585]: E0709 18:50:41.034548 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/235c900d-b372-41f9-bf33-0f1a5bf90e63-maas-api-tls podName:235c900d-b372-41f9-bf33-0f1a5bf90e63 nodeName:}" failed. No retries permitted until 2026-07-09 18:50:41.534528834 +0000 UTC m=+558.278511931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/235c900d-b372-41f9-bf33-0f1a5bf90e63-maas-api-tls") pod "maas-api-d89b88c4c-ccnjk" (UID: "235c900d-b372-41f9-bf33-0f1a5bf90e63") : secret "maas-api-serving-cert" not found Jul 09 18:50:41.034997 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.034981 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/235c900d-b372-41f9-bf33-0f1a5bf90e63-openshift-service-ca\") pod \"maas-api-d89b88c4c-ccnjk\" (UID: \"235c900d-b372-41f9-bf33-0f1a5bf90e63\") " pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" Jul 09 18:50:41.043867 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.043847 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8fz5\" (UniqueName: \"kubernetes.io/projected/235c900d-b372-41f9-bf33-0f1a5bf90e63-kube-api-access-k8fz5\") pod \"maas-api-d89b88c4c-ccnjk\" (UID: \"235c900d-b372-41f9-bf33-0f1a5bf90e63\") " pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" Jul 09 18:50:41.419034 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.419002 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-v2qj2"] Jul 09 18:50:41.422509 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.422491 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-v2qj2" Jul 09 18:50:41.425109 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.425087 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 18:50:41.433224 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.433203 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-v2qj2"] Jul 09 18:50:41.448103 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.448078 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-bh4h4"] Jul 09 18:50:41.451305 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.451291 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-bh4h4" Jul 09 18:50:41.453967 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.453953 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4xvgk\"" Jul 09 18:50:41.472275 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.472249 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-bh4h4"] Jul 09 18:50:41.539166 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.539127 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235c900d-b372-41f9-bf33-0f1a5bf90e63-maas-api-tls\") pod \"maas-api-d89b88c4c-ccnjk\" (UID: \"235c900d-b372-41f9-bf33-0f1a5bf90e63\") " pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" Jul 09 18:50:41.539363 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.539212 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/431f617b-dd44-4b8d-80f8-ee8cfc290998-plugins-config-volume\") pod \"payload-processing-58f86c7798-bh4h4\" (UID: \"431f617b-dd44-4b8d-80f8-ee8cfc290998\") " pod="openshift-ingress/payload-processing-58f86c7798-bh4h4" Jul 09 18:50:41.539363 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.539240 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbbjb\" (UniqueName: \"kubernetes.io/projected/431f617b-dd44-4b8d-80f8-ee8cfc290998-kube-api-access-lbbjb\") pod \"payload-processing-58f86c7798-bh4h4\" (UID: \"431f617b-dd44-4b8d-80f8-ee8cfc290998\") " pod="openshift-ingress/payload-processing-58f86c7798-bh4h4" Jul 09 18:50:41.539363 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.539298 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gphk7\" (UniqueName: \"kubernetes.io/projected/c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d-kube-api-access-gphk7\") pod \"payload-pre-processing-7964457696-v2qj2\" (UID: \"c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d\") " pod="openshift-ingress/payload-pre-processing-7964457696-v2qj2" Jul 09 18:50:41.539363 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.539323 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d-plugins-config-volume\") pod \"payload-pre-processing-7964457696-v2qj2\" (UID: \"c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d\") " pod="openshift-ingress/payload-pre-processing-7964457696-v2qj2" Jul 09 18:50:41.541755 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.541731 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235c900d-b372-41f9-bf33-0f1a5bf90e63-maas-api-tls\") pod \"maas-api-d89b88c4c-ccnjk\" (UID: \"235c900d-b372-41f9-bf33-0f1a5bf90e63\") " pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" Jul 09 18:50:41.640196 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.640154 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/431f617b-dd44-4b8d-80f8-ee8cfc290998-plugins-config-volume\") pod \"payload-processing-58f86c7798-bh4h4\" (UID: \"431f617b-dd44-4b8d-80f8-ee8cfc290998\") " pod="openshift-ingress/payload-processing-58f86c7798-bh4h4" Jul 09 18:50:41.640196 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.640199 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbbjb\" (UniqueName: \"kubernetes.io/projected/431f617b-dd44-4b8d-80f8-ee8cfc290998-kube-api-access-lbbjb\") pod \"payload-processing-58f86c7798-bh4h4\" (UID: \"431f617b-dd44-4b8d-80f8-ee8cfc290998\") " pod="openshift-ingress/payload-processing-58f86c7798-bh4h4" Jul 09 18:50:41.640451 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.640394 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gphk7\" (UniqueName: \"kubernetes.io/projected/c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d-kube-api-access-gphk7\") pod \"payload-pre-processing-7964457696-v2qj2\" (UID: \"c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d\") " pod="openshift-ingress/payload-pre-processing-7964457696-v2qj2" Jul 09 18:50:41.640451 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.640441 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d-plugins-config-volume\") pod \"payload-pre-processing-7964457696-v2qj2\" (UID: \"c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d\") " pod="openshift-ingress/payload-pre-processing-7964457696-v2qj2" Jul 09 18:50:41.641134 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.641108 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d-plugins-config-volume\") pod \"payload-pre-processing-7964457696-v2qj2\" (UID: \"c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d\") " pod="openshift-ingress/payload-pre-processing-7964457696-v2qj2" Jul 09 18:50:41.641256 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.641145 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/431f617b-dd44-4b8d-80f8-ee8cfc290998-plugins-config-volume\") pod \"payload-processing-58f86c7798-bh4h4\" (UID: \"431f617b-dd44-4b8d-80f8-ee8cfc290998\") " pod="openshift-ingress/payload-processing-58f86c7798-bh4h4" Jul 09 18:50:41.651919 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.651880 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbbjb\" (UniqueName: \"kubernetes.io/projected/431f617b-dd44-4b8d-80f8-ee8cfc290998-kube-api-access-lbbjb\") pod \"payload-processing-58f86c7798-bh4h4\" (UID: \"431f617b-dd44-4b8d-80f8-ee8cfc290998\") " pod="openshift-ingress/payload-processing-58f86c7798-bh4h4" Jul 09 18:50:41.653285 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.653262 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gphk7\" (UniqueName: \"kubernetes.io/projected/c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d-kube-api-access-gphk7\") pod \"payload-pre-processing-7964457696-v2qj2\" (UID: \"c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d\") " pod="openshift-ingress/payload-pre-processing-7964457696-v2qj2" Jul 09 18:50:41.733846 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.733764 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-v2qj2" Jul 09 18:50:41.761265 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.761236 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-bh4h4" Jul 09 18:50:41.810982 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.810939 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" Jul 09 18:50:41.894841 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.894413 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-v2qj2"] Jul 09 18:50:41.896926 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:50:41.896868 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00f0b7e_6103_4cbb_9d62_4ae4569e0f5d.slice/crio-2c0a7c68ac310870039427e6bf45a01363bbab5b591e59294f4e43ea693c0205 WatchSource:0}: Error finding container 2c0a7c68ac310870039427e6bf45a01363bbab5b591e59294f4e43ea693c0205: Status 404 returned error can't find the container with id 2c0a7c68ac310870039427e6bf45a01363bbab5b591e59294f4e43ea693c0205 Jul 09 18:50:41.926183 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.926114 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-bh4h4"] Jul 09 18:50:41.929364 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:50:41.929329 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod431f617b_dd44_4b8d_80f8_ee8cfc290998.slice/crio-60df241d27172bc4ae7ca5277be81b306e64f41f0f39c2ddd0a998555d3d2c97 WatchSource:0}: Error finding container 60df241d27172bc4ae7ca5277be81b306e64f41f0f39c2ddd0a998555d3d2c97: Status 404 returned error can't find the container with id 60df241d27172bc4ae7ca5277be81b306e64f41f0f39c2ddd0a998555d3d2c97 Jul 09 18:50:41.970702 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:41.970680 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk"] Jul 09 18:50:41.972543 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:50:41.972518 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod235c900d_b372_41f9_bf33_0f1a5bf90e63.slice/crio-687e306648f6b7212b8ccd5e3f87ab8ae5e444ef35f115a13e7b01b6e5574a28 WatchSource:0}: Error finding container 687e306648f6b7212b8ccd5e3f87ab8ae5e444ef35f115a13e7b01b6e5574a28: Status 404 returned error can't find the container with id 687e306648f6b7212b8ccd5e3f87ab8ae5e444ef35f115a13e7b01b6e5574a28 Jul 09 18:50:42.388782 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:42.388744 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" event={"ID":"235c900d-b372-41f9-bf33-0f1a5bf90e63","Type":"ContainerStarted","Data":"687e306648f6b7212b8ccd5e3f87ab8ae5e444ef35f115a13e7b01b6e5574a28"} Jul 09 18:50:42.390003 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:42.389975 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-bh4h4" event={"ID":"431f617b-dd44-4b8d-80f8-ee8cfc290998","Type":"ContainerStarted","Data":"60df241d27172bc4ae7ca5277be81b306e64f41f0f39c2ddd0a998555d3d2c97"} Jul 09 18:50:42.391069 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:42.391049 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-v2qj2" event={"ID":"c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d","Type":"ContainerStarted","Data":"2c0a7c68ac310870039427e6bf45a01363bbab5b591e59294f4e43ea693c0205"} Jul 09 18:50:44.403462 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:44.403360 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-bh4h4" event={"ID":"431f617b-dd44-4b8d-80f8-ee8cfc290998","Type":"ContainerStarted","Data":"84defd2a35f3de67de1e729d308058bc5d0ac784554a70ad7fbaab51520db2c7"} Jul 09 18:50:44.403922 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:44.403503 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-bh4h4" Jul 09 18:50:44.404951 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:44.404926 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-v2qj2" event={"ID":"c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d","Type":"ContainerStarted","Data":"cf615467a62804bf71bfe54327fa950dbf021462dd29ae676ea52522e9cd5153"} Jul 09 18:50:44.405076 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:44.404973 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-v2qj2" Jul 09 18:50:44.406349 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:44.406327 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" event={"ID":"235c900d-b372-41f9-bf33-0f1a5bf90e63","Type":"ContainerStarted","Data":"87834c3647fb304a99cda4079ea7851760788d1443f325d1f9a28dc09c401256"} Jul 09 18:50:44.406483 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:44.406455 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" Jul 09 18:50:44.422100 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:44.422052 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-bh4h4" podStartSLOduration=1.99272084 podStartE2EDuration="3.422038495s" totalImagesPullingTime="1.429317655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:50:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:50:44.420026862 +0000 UTC m=+561.164009978" watchObservedRunningTime="2026-07-09 18:50:44.422038495 +0000 UTC m=+561.166021641" Jul 09 18:50:44.439862 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:44.439816 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" podStartSLOduration=2.442689318 podStartE2EDuration="4.439800426s" totalImagesPullingTime="1.997111108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:50:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:50:44.438860863 +0000 UTC m=+561.182843971" watchObservedRunningTime="2026-07-09 18:50:44.439800426 +0000 UTC m=+561.183783539" Jul 09 18:50:44.456370 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:44.456324 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-v2qj2" podStartSLOduration=1.994386865 podStartE2EDuration="3.456311362s" totalImagesPullingTime="1.461924497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:50:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:50:44.455551274 +0000 UTC m=+561.199534389" watchObservedRunningTime="2026-07-09 18:50:44.456311362 +0000 UTC m=+561.200294665" Jul 09 18:50:50.416069 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:50.416043 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-d89b88c4c-ccnjk" Jul 09 18:50:55.412706 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:55.412677 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-v2qj2" Jul 09 18:50:55.413156 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:50:55.412853 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-bh4h4" Jul 09 18:51:10.171990 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.171961 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc"] Jul 09 18:51:10.175776 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.175756 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.178507 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.178485 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 18:51:10.179405 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.179388 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 18:51:10.179507 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.179440 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gqjpx\"" Jul 09 18:51:10.179507 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.179451 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 09 18:51:10.187082 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.187062 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc"] Jul 09 18:51:10.293091 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.293056 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9drs\" (UniqueName: \"kubernetes.io/projected/851b617b-168b-41b4-96db-8ca8cf61257d-kube-api-access-j9drs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.293255 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.293134 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/851b617b-168b-41b4-96db-8ca8cf61257d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.293255 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.293173 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/851b617b-168b-41b4-96db-8ca8cf61257d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.293255 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.293192 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/851b617b-168b-41b4-96db-8ca8cf61257d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.293255 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.293208 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/851b617b-168b-41b4-96db-8ca8cf61257d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.293255 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.293223 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/851b617b-168b-41b4-96db-8ca8cf61257d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.395811 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.395780 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9drs\" (UniqueName: \"kubernetes.io/projected/851b617b-168b-41b4-96db-8ca8cf61257d-kube-api-access-j9drs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.396006 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.395947 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/851b617b-168b-41b4-96db-8ca8cf61257d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.396006 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.395999 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/851b617b-168b-41b4-96db-8ca8cf61257d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.396136 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.396031 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/851b617b-168b-41b4-96db-8ca8cf61257d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.396136 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.396047 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/851b617b-168b-41b4-96db-8ca8cf61257d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.396136 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.396063 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/851b617b-168b-41b4-96db-8ca8cf61257d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.396460 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.396431 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/851b617b-168b-41b4-96db-8ca8cf61257d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.396548 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.396493 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/851b617b-168b-41b4-96db-8ca8cf61257d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.396548 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.396535 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/851b617b-168b-41b4-96db-8ca8cf61257d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.398667 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.398634 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/851b617b-168b-41b4-96db-8ca8cf61257d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.398895 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.398880 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/851b617b-168b-41b4-96db-8ca8cf61257d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.405054 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.405035 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9drs\" (UniqueName: \"kubernetes.io/projected/851b617b-168b-41b4-96db-8ca8cf61257d-kube-api-access-j9drs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dhmmc\" (UID: \"851b617b-168b-41b4-96db-8ca8cf61257d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.487040 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.486950 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:10.633727 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:10.633694 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc"] Jul 09 18:51:10.636448 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:51:10.636417 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod851b617b_168b_41b4_96db_8ca8cf61257d.slice/crio-eb94c875035af671bf3569bed0399b49de3ad76f923baff6bf0ffcef76fb6168 WatchSource:0}: Error finding container eb94c875035af671bf3569bed0399b49de3ad76f923baff6bf0ffcef76fb6168: Status 404 returned error can't find the container with id eb94c875035af671bf3569bed0399b49de3ad76f923baff6bf0ffcef76fb6168 Jul 09 18:51:11.527455 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:11.527414 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" event={"ID":"851b617b-168b-41b4-96db-8ca8cf61257d","Type":"ContainerStarted","Data":"eb94c875035af671bf3569bed0399b49de3ad76f923baff6bf0ffcef76fb6168"} Jul 09 18:51:12.533661 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:12.533627 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" event={"ID":"851b617b-168b-41b4-96db-8ca8cf61257d","Type":"ContainerStarted","Data":"e33bc387c0ec220b0f4ec6c97d78a826d153f6fc74bee3b14a081686c9645770"} Jul 09 18:51:12.557749 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:12.557697 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" podStartSLOduration=1.42642918 podStartE2EDuration="2.557680274s" totalImagesPullingTime="1.131251094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:51:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:51:12.556044064 +0000 UTC m=+589.300027180" watchObservedRunningTime="2026-07-09 18:51:12.557680274 +0000 UTC m=+589.301663389" Jul 09 18:51:20.488086 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:20.488045 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:20.488086 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:20.488100 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:20.500779 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:20.500753 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:20.580907 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:20.580881 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dhmmc" Jul 09 18:51:24.397366 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:24.397256 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 18:51:24.397366 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:24.397265 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 18:51:57.753358 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.753281 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw"] Jul 09 18:51:57.757366 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.757346 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.760164 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.760140 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 09 18:51:57.770341 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.770320 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw"] Jul 09 18:51:57.820004 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.819975 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4db54f1b-62a6-4145-b525-cd84723a0e8e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.820162 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.820013 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4db54f1b-62a6-4145-b525-cd84723a0e8e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.820162 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.820078 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4db54f1b-62a6-4145-b525-cd84723a0e8e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.820162 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.820105 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4db54f1b-62a6-4145-b525-cd84723a0e8e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.820162 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.820128 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4db54f1b-62a6-4145-b525-cd84723a0e8e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.820162 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.820147 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6m5q\" (UniqueName: \"kubernetes.io/projected/4db54f1b-62a6-4145-b525-cd84723a0e8e-kube-api-access-w6m5q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.921241 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.921206 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4db54f1b-62a6-4145-b525-cd84723a0e8e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.921241 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.921247 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4db54f1b-62a6-4145-b525-cd84723a0e8e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.921483 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.921268 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6m5q\" (UniqueName: \"kubernetes.io/projected/4db54f1b-62a6-4145-b525-cd84723a0e8e-kube-api-access-w6m5q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.921483 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.921316 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4db54f1b-62a6-4145-b525-cd84723a0e8e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.921483 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.921336 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4db54f1b-62a6-4145-b525-cd84723a0e8e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.921483 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.921370 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4db54f1b-62a6-4145-b525-cd84723a0e8e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.925609 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.921931 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4db54f1b-62a6-4145-b525-cd84723a0e8e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.925609 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.921997 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4db54f1b-62a6-4145-b525-cd84723a0e8e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.925609 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.922095 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4db54f1b-62a6-4145-b525-cd84723a0e8e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.925609 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.924985 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4db54f1b-62a6-4145-b525-cd84723a0e8e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.925908 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.925772 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4db54f1b-62a6-4145-b525-cd84723a0e8e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:57.931948 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:57.931923 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6m5q\" (UniqueName: \"kubernetes.io/projected/4db54f1b-62a6-4145-b525-cd84723a0e8e-kube-api-access-w6m5q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw\" (UID: \"4db54f1b-62a6-4145-b525-cd84723a0e8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:58.068258 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:58.068172 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:51:58.202506 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:58.202448 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw"] Jul 09 18:51:58.204917 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:51:58.204889 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4db54f1b_62a6_4145_b525_cd84723a0e8e.slice/crio-695c09bd71fe2c52047c0cfc55cde76de59ac6023beb226bc6760d567efc87ec WatchSource:0}: Error finding container 695c09bd71fe2c52047c0cfc55cde76de59ac6023beb226bc6760d567efc87ec: Status 404 returned error can't find the container with id 695c09bd71fe2c52047c0cfc55cde76de59ac6023beb226bc6760d567efc87ec Jul 09 18:51:58.206318 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:58.206302 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 18:51:58.747889 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:58.747855 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" event={"ID":"4db54f1b-62a6-4145-b525-cd84723a0e8e","Type":"ContainerStarted","Data":"af61a5955b1a11f82bd49ad22b59e59d55093f06d44d3334dbe3f20962d30a1e"} Jul 09 18:51:58.748103 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:58.747906 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" event={"ID":"4db54f1b-62a6-4145-b525-cd84723a0e8e","Type":"ContainerStarted","Data":"695c09bd71fe2c52047c0cfc55cde76de59ac6023beb226bc6760d567efc87ec"} Jul 09 18:51:58.769926 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:51:58.769875 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" podStartSLOduration=1.592952015 podStartE2EDuration="1.769860205s" totalImagesPullingTime="176.90819ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:51:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:51:58.768208507 +0000 UTC m=+635.512191622" watchObservedRunningTime="2026-07-09 18:51:58.769860205 +0000 UTC m=+635.513843339" Jul 09 18:52:08.068564 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:52:08.068527 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:52:08.069032 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:52:08.068615 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:52:08.081206 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:52:08.081176 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:52:08.807538 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:52:08.807508 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw" Jul 09 18:53:25.453993 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:25.453906 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55cf685454-bzw97"] Jul 09 18:53:25.457290 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:25.457273 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55cf685454-bzw97" Jul 09 18:53:25.459871 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:25.459853 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 18:53:25.460983 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:25.460962 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xp8dl\"" Jul 09 18:53:25.461097 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:25.461011 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 18:53:25.461097 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:25.461044 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 09 18:53:25.467132 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:25.467108 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55cf685454-bzw97"] Jul 09 18:53:25.572886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:25.572854 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/43f096dd-39ad-4ca5-a8c0-315345407e32-tls-cert\") pod \"authorino-55cf685454-bzw97\" (UID: \"43f096dd-39ad-4ca5-a8c0-315345407e32\") " pod="kuadrant-system/authorino-55cf685454-bzw97" Jul 09 18:53:25.572886 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:25.572894 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rqcs\" (UniqueName: \"kubernetes.io/projected/43f096dd-39ad-4ca5-a8c0-315345407e32-kube-api-access-8rqcs\") pod \"authorino-55cf685454-bzw97\" (UID: \"43f096dd-39ad-4ca5-a8c0-315345407e32\") " pod="kuadrant-system/authorino-55cf685454-bzw97" Jul 09 18:53:25.673929 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:25.673887 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/43f096dd-39ad-4ca5-a8c0-315345407e32-tls-cert\") pod \"authorino-55cf685454-bzw97\" (UID: \"43f096dd-39ad-4ca5-a8c0-315345407e32\") " pod="kuadrant-system/authorino-55cf685454-bzw97" Jul 09 18:53:25.674097 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:25.673939 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rqcs\" (UniqueName: \"kubernetes.io/projected/43f096dd-39ad-4ca5-a8c0-315345407e32-kube-api-access-8rqcs\") pod \"authorino-55cf685454-bzw97\" (UID: \"43f096dd-39ad-4ca5-a8c0-315345407e32\") " pod="kuadrant-system/authorino-55cf685454-bzw97" Jul 09 18:53:25.676627 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:25.676607 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/43f096dd-39ad-4ca5-a8c0-315345407e32-tls-cert\") pod \"authorino-55cf685454-bzw97\" (UID: \"43f096dd-39ad-4ca5-a8c0-315345407e32\") " pod="kuadrant-system/authorino-55cf685454-bzw97" Jul 09 18:53:25.684333 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:25.684303 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rqcs\" (UniqueName: \"kubernetes.io/projected/43f096dd-39ad-4ca5-a8c0-315345407e32-kube-api-access-8rqcs\") pod \"authorino-55cf685454-bzw97\" (UID: \"43f096dd-39ad-4ca5-a8c0-315345407e32\") " pod="kuadrant-system/authorino-55cf685454-bzw97" Jul 09 18:53:25.767083 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:25.767006 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55cf685454-bzw97" Jul 09 18:53:25.909205 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:25.909178 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55cf685454-bzw97"] Jul 09 18:53:25.910554 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:53:25.910525 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43f096dd_39ad_4ca5_a8c0_315345407e32.slice/crio-4ec2c46d7c4b480ea0a0004a2cf7e69c2e43b8b71d068f4c6763ec4d49bdea01 WatchSource:0}: Error finding container 4ec2c46d7c4b480ea0a0004a2cf7e69c2e43b8b71d068f4c6763ec4d49bdea01: Status 404 returned error can't find the container with id 4ec2c46d7c4b480ea0a0004a2cf7e69c2e43b8b71d068f4c6763ec4d49bdea01 Jul 09 18:53:26.140525 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:26.140444 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55cf685454-bzw97" event={"ID":"43f096dd-39ad-4ca5-a8c0-315345407e32","Type":"ContainerStarted","Data":"4ec2c46d7c4b480ea0a0004a2cf7e69c2e43b8b71d068f4c6763ec4d49bdea01"} Jul 09 18:53:27.148432 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:27.148385 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55cf685454-bzw97" event={"ID":"43f096dd-39ad-4ca5-a8c0-315345407e32","Type":"ContainerStarted","Data":"e8170dc2836d54077b303c0277681a256f8a714445be1fd1665fb30ef6d610dc"} Jul 09 18:53:27.205171 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:53:27.205121 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-55cf685454-bzw97" podStartSLOduration=1.908129847 podStartE2EDuration="2.20510357s" totalImagesPullingTime="296.973723ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:53:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:53:27.203173923 +0000 UTC m=+723.947157045" watchObservedRunningTime="2026-07-09 18:53:27.20510357 +0000 UTC m=+723.949086684" Jul 09 18:54:38.321501 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:38.321464 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f5cd9bc7b-7cwpd"] Jul 09 18:54:38.322082 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:38.321707 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" podUID="79c8dbb5-a8b8-4cf8-aa0f-0b388daae041" containerName="manager" containerID="cri-o://554afc719c1a57dd6d6ab11f516bb790d4cf6b413e45d212f4aafa116706dadf" gracePeriod=30 Jul 09 18:54:38.475232 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:38.475200 2585 generic.go:356] "Generic (PLEG): container finished" podID="79c8dbb5-a8b8-4cf8-aa0f-0b388daae041" containerID="554afc719c1a57dd6d6ab11f516bb790d4cf6b413e45d212f4aafa116706dadf" exitCode=0 Jul 09 18:54:38.475418 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:38.475272 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" event={"ID":"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041","Type":"ContainerDied","Data":"554afc719c1a57dd6d6ab11f516bb790d4cf6b413e45d212f4aafa116706dadf"} Jul 09 18:54:38.570697 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:38.570674 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" Jul 09 18:54:38.625057 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:38.624985 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-service-ca\") pod \"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041\" (UID: \"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041\") " Jul 09 18:54:38.625057 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:38.625053 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-webhook-cert\") pod \"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041\" (UID: \"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041\") " Jul 09 18:54:38.625277 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:38.625103 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t76mj\" (UniqueName: \"kubernetes.io/projected/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-kube-api-access-t76mj\") pod \"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041\" (UID: \"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041\") " Jul 09 18:54:38.625277 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:38.625211 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-service-ca" (OuterVolumeSpecName: "service-ca") pod "79c8dbb5-a8b8-4cf8-aa0f-0b388daae041" (UID: "79c8dbb5-a8b8-4cf8-aa0f-0b388daae041"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:54:38.625409 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:38.625392 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:54:38.627230 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:38.627203 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-kube-api-access-t76mj" (OuterVolumeSpecName: "kube-api-access-t76mj") pod "79c8dbb5-a8b8-4cf8-aa0f-0b388daae041" (UID: "79c8dbb5-a8b8-4cf8-aa0f-0b388daae041"). InnerVolumeSpecName "kube-api-access-t76mj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:54:38.627338 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:38.627302 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "79c8dbb5-a8b8-4cf8-aa0f-0b388daae041" (UID: "79c8dbb5-a8b8-4cf8-aa0f-0b388daae041"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:54:38.726645 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:38.726602 2585 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-webhook-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:54:38.726645 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:38.726642 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t76mj\" (UniqueName: \"kubernetes.io/projected/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041-kube-api-access-t76mj\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 18:54:39.480797 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:39.480766 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" event={"ID":"79c8dbb5-a8b8-4cf8-aa0f-0b388daae041","Type":"ContainerDied","Data":"f5eb578209a37b1b58250be666a3c97ba2943465e98d4bf975451b47cc7d2d5b"} Jul 09 18:54:39.480797 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:39.480798 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5cd9bc7b-7cwpd" Jul 09 18:54:39.481289 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:39.480822 2585 scope.go:117] "RemoveContainer" containerID="554afc719c1a57dd6d6ab11f516bb790d4cf6b413e45d212f4aafa116706dadf" Jul 09 18:54:39.512753 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:39.512718 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f5cd9bc7b-7cwpd"] Jul 09 18:54:39.522916 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:39.522883 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f5cd9bc7b-7cwpd"] Jul 09 18:54:39.847379 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:39.847296 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79c8dbb5-a8b8-4cf8-aa0f-0b388daae041" path="/var/lib/kubelet/pods/79c8dbb5-a8b8-4cf8-aa0f-0b388daae041/volumes" Jul 09 18:54:40.313497 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.313467 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f5cd9bc7b-5tf2w"] Jul 09 18:54:40.313868 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.313852 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79c8dbb5-a8b8-4cf8-aa0f-0b388daae041" containerName="manager" Jul 09 18:54:40.313868 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.313869 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="79c8dbb5-a8b8-4cf8-aa0f-0b388daae041" containerName="manager" Jul 09 18:54:40.313955 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.313940 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="79c8dbb5-a8b8-4cf8-aa0f-0b388daae041" containerName="manager" Jul 09 18:54:40.318549 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.318531 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" Jul 09 18:54:40.321684 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.321661 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-gmrv2\"" Jul 09 18:54:40.322516 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.322499 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 18:54:40.322642 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.322542 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 18:54:40.327457 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.327433 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f5cd9bc7b-5tf2w"] Jul 09 18:54:40.341932 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.341909 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf-service-ca\") pod \"maas-controller-5f5cd9bc7b-5tf2w\" (UID: \"ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf\") " pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" Jul 09 18:54:40.342035 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.341974 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxw97\" (UniqueName: \"kubernetes.io/projected/ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf-kube-api-access-dxw97\") pod \"maas-controller-5f5cd9bc7b-5tf2w\" (UID: \"ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf\") " pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" Jul 09 18:54:40.342084 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.342051 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf-webhook-cert\") pod \"maas-controller-5f5cd9bc7b-5tf2w\" (UID: \"ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf\") " pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" Jul 09 18:54:40.442871 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.442847 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxw97\" (UniqueName: \"kubernetes.io/projected/ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf-kube-api-access-dxw97\") pod \"maas-controller-5f5cd9bc7b-5tf2w\" (UID: \"ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf\") " pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" Jul 09 18:54:40.443013 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.442895 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf-webhook-cert\") pod \"maas-controller-5f5cd9bc7b-5tf2w\" (UID: \"ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf\") " pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" Jul 09 18:54:40.443013 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.442921 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf-service-ca\") pod \"maas-controller-5f5cd9bc7b-5tf2w\" (UID: \"ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf\") " pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" Jul 09 18:54:40.443698 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.443550 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf-service-ca\") pod \"maas-controller-5f5cd9bc7b-5tf2w\" (UID: \"ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf\") " pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" Jul 09 18:54:40.445427 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.445405 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf-webhook-cert\") pod \"maas-controller-5f5cd9bc7b-5tf2w\" (UID: \"ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf\") " pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" Jul 09 18:54:40.453515 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.453489 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxw97\" (UniqueName: \"kubernetes.io/projected/ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf-kube-api-access-dxw97\") pod \"maas-controller-5f5cd9bc7b-5tf2w\" (UID: \"ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf\") " pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" Jul 09 18:54:40.628921 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.628832 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" Jul 09 18:54:40.766284 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:40.766261 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f5cd9bc7b-5tf2w"] Jul 09 18:54:40.768418 ip-10-0-143-175 kubenswrapper[2585]: W0709 18:54:40.768387 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac55a28b_cd8b_4bcf_b714_f50b2a2d0ecf.slice/crio-7ca419f9411e2c93aaff865b4e8e60e0fa70bc4bff4e6365972cb54e2f30ea69 WatchSource:0}: Error finding container 7ca419f9411e2c93aaff865b4e8e60e0fa70bc4bff4e6365972cb54e2f30ea69: Status 404 returned error can't find the container with id 7ca419f9411e2c93aaff865b4e8e60e0fa70bc4bff4e6365972cb54e2f30ea69 Jul 09 18:54:41.491818 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:41.491730 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" event={"ID":"ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf","Type":"ContainerStarted","Data":"14f90ab01921b4eba2e11fb2ebaf6777e654616f9f5ae1e1d2df8eee9afdbfc7"} Jul 09 18:54:41.491818 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:41.491787 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" event={"ID":"ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf","Type":"ContainerStarted","Data":"7ca419f9411e2c93aaff865b4e8e60e0fa70bc4bff4e6365972cb54e2f30ea69"} Jul 09 18:54:41.491818 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:41.491817 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" Jul 09 18:54:41.511233 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:41.511182 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" podStartSLOduration=1.077373501 podStartE2EDuration="1.511166391s" totalImagesPullingTime="433.79289ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:54:41.510745751 +0000 UTC m=+798.254728864" watchObservedRunningTime="2026-07-09 18:54:41.511166391 +0000 UTC m=+798.255149503" Jul 09 18:54:52.502528 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:54:52.502458 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f5cd9bc7b-5tf2w" Jul 09 18:56:24.424926 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:56:24.424897 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 18:56:24.427247 ip-10-0-143-175 kubenswrapper[2585]: I0709 18:56:24.426446 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 19:01:24.455025 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:01:24.454913 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 19:01:24.455025 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:01:24.454955 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 19:06:24.482505 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:24.482393 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 19:06:24.486467 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:24.485121 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 19:06:42.799830 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:42.799799 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72"] Jul 09 19:06:42.803221 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:42.803195 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" Jul 09 19:06:42.807223 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:42.807202 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-231d4590-serving-cert\"" Jul 09 19:06:42.817378 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:42.817355 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72"] Jul 09 19:06:42.845423 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:42.845393 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265bd5cb-de44-4eed-bff2-e528cd50c922-openshift-service-ca\") pod \"maas-api-e2e-ait-231d4590-645c657cb6-nfk72\" (UID: \"265bd5cb-de44-4eed-bff2-e528cd50c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" Jul 09 19:06:42.845569 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:42.845511 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2dz8\" (UniqueName: \"kubernetes.io/projected/265bd5cb-de44-4eed-bff2-e528cd50c922-kube-api-access-z2dz8\") pod \"maas-api-e2e-ait-231d4590-645c657cb6-nfk72\" (UID: \"265bd5cb-de44-4eed-bff2-e528cd50c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" Jul 09 19:06:42.845641 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:42.845576 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265bd5cb-de44-4eed-bff2-e528cd50c922-maas-api-tls\") pod \"maas-api-e2e-ait-231d4590-645c657cb6-nfk72\" (UID: \"265bd5cb-de44-4eed-bff2-e528cd50c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" Jul 09 19:06:42.946192 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:42.946161 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2dz8\" (UniqueName: \"kubernetes.io/projected/265bd5cb-de44-4eed-bff2-e528cd50c922-kube-api-access-z2dz8\") pod \"maas-api-e2e-ait-231d4590-645c657cb6-nfk72\" (UID: \"265bd5cb-de44-4eed-bff2-e528cd50c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" Jul 09 19:06:42.946368 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:42.946333 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265bd5cb-de44-4eed-bff2-e528cd50c922-maas-api-tls\") pod \"maas-api-e2e-ait-231d4590-645c657cb6-nfk72\" (UID: \"265bd5cb-de44-4eed-bff2-e528cd50c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" Jul 09 19:06:42.946430 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:42.946385 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265bd5cb-de44-4eed-bff2-e528cd50c922-openshift-service-ca\") pod \"maas-api-e2e-ait-231d4590-645c657cb6-nfk72\" (UID: \"265bd5cb-de44-4eed-bff2-e528cd50c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" Jul 09 19:06:42.946989 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:42.946967 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265bd5cb-de44-4eed-bff2-e528cd50c922-openshift-service-ca\") pod \"maas-api-e2e-ait-231d4590-645c657cb6-nfk72\" (UID: \"265bd5cb-de44-4eed-bff2-e528cd50c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" Jul 09 19:06:42.948913 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:42.948893 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265bd5cb-de44-4eed-bff2-e528cd50c922-maas-api-tls\") pod \"maas-api-e2e-ait-231d4590-645c657cb6-nfk72\" (UID: \"265bd5cb-de44-4eed-bff2-e528cd50c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" Jul 09 19:06:42.955989 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:42.955967 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2dz8\" (UniqueName: \"kubernetes.io/projected/265bd5cb-de44-4eed-bff2-e528cd50c922-kube-api-access-z2dz8\") pod \"maas-api-e2e-ait-231d4590-645c657cb6-nfk72\" (UID: \"265bd5cb-de44-4eed-bff2-e528cd50c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" Jul 09 19:06:43.117573 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:43.117476 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" Jul 09 19:06:43.264836 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:43.264783 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72"] Jul 09 19:06:43.267191 ip-10-0-143-175 kubenswrapper[2585]: W0709 19:06:43.267156 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod265bd5cb_de44_4eed_bff2_e528cd50c922.slice/crio-df50f08fe39d76070ca5538f480033aaaffe471c4a7722942054a6d8a79f31b1 WatchSource:0}: Error finding container df50f08fe39d76070ca5538f480033aaaffe471c4a7722942054a6d8a79f31b1: Status 404 returned error can't find the container with id df50f08fe39d76070ca5538f480033aaaffe471c4a7722942054a6d8a79f31b1 Jul 09 19:06:43.268428 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:43.268410 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 19:06:43.811576 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:43.811548 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" event={"ID":"265bd5cb-de44-4eed-bff2-e528cd50c922","Type":"ContainerStarted","Data":"11cad6f4c62fb9133d7e6043742c3b47b57479884802c594b39f38c4d75c004b"} Jul 09 19:06:43.812015 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:43.811620 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" event={"ID":"265bd5cb-de44-4eed-bff2-e528cd50c922","Type":"ContainerStarted","Data":"df50f08fe39d76070ca5538f480033aaaffe471c4a7722942054a6d8a79f31b1"} Jul 09 19:06:43.812015 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:43.811653 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" Jul 09 19:06:43.831411 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:43.831365 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" podStartSLOduration=1.356510107 podStartE2EDuration="1.831349437s" totalImagesPullingTime="474.83933ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:06:43.829623123 +0000 UTC m=+1520.573606236" watchObservedRunningTime="2026-07-09 19:06:43.831349437 +0000 UTC m=+1520.575332552" Jul 09 19:06:49.822266 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:49.822239 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" Jul 09 19:06:50.716652 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:50.716619 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72"] Jul 09 19:06:50.716872 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:50.716835 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" podUID="265bd5cb-de44-4eed-bff2-e528cd50c922" containerName="maas-api" containerID="cri-o://11cad6f4c62fb9133d7e6043742c3b47b57479884802c594b39f38c4d75c004b" gracePeriod=30 Jul 09 19:06:50.845563 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:50.845528 2585 generic.go:356] "Generic (PLEG): container finished" podID="265bd5cb-de44-4eed-bff2-e528cd50c922" containerID="11cad6f4c62fb9133d7e6043742c3b47b57479884802c594b39f38c4d75c004b" exitCode=0 Jul 09 19:06:50.846182 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:50.846150 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" event={"ID":"265bd5cb-de44-4eed-bff2-e528cd50c922","Type":"ContainerDied","Data":"11cad6f4c62fb9133d7e6043742c3b47b57479884802c594b39f38c4d75c004b"} Jul 09 19:06:50.976961 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:50.976903 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" Jul 09 19:06:51.020389 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:51.020364 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265bd5cb-de44-4eed-bff2-e528cd50c922-maas-api-tls\") pod \"265bd5cb-de44-4eed-bff2-e528cd50c922\" (UID: \"265bd5cb-de44-4eed-bff2-e528cd50c922\") " Jul 09 19:06:51.020533 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:51.020433 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265bd5cb-de44-4eed-bff2-e528cd50c922-openshift-service-ca\") pod \"265bd5cb-de44-4eed-bff2-e528cd50c922\" (UID: \"265bd5cb-de44-4eed-bff2-e528cd50c922\") " Jul 09 19:06:51.020576 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:51.020532 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2dz8\" (UniqueName: \"kubernetes.io/projected/265bd5cb-de44-4eed-bff2-e528cd50c922-kube-api-access-z2dz8\") pod \"265bd5cb-de44-4eed-bff2-e528cd50c922\" (UID: \"265bd5cb-de44-4eed-bff2-e528cd50c922\") " Jul 09 19:06:51.020878 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:51.020841 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/265bd5cb-de44-4eed-bff2-e528cd50c922-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "265bd5cb-de44-4eed-bff2-e528cd50c922" (UID: "265bd5cb-de44-4eed-bff2-e528cd50c922"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:06:51.022866 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:51.022838 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/265bd5cb-de44-4eed-bff2-e528cd50c922-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "265bd5cb-de44-4eed-bff2-e528cd50c922" (UID: "265bd5cb-de44-4eed-bff2-e528cd50c922"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:06:51.022972 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:51.022861 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/265bd5cb-de44-4eed-bff2-e528cd50c922-kube-api-access-z2dz8" (OuterVolumeSpecName: "kube-api-access-z2dz8") pod "265bd5cb-de44-4eed-bff2-e528cd50c922" (UID: "265bd5cb-de44-4eed-bff2-e528cd50c922"). InnerVolumeSpecName "kube-api-access-z2dz8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:06:51.121820 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:51.121790 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2dz8\" (UniqueName: \"kubernetes.io/projected/265bd5cb-de44-4eed-bff2-e528cd50c922-kube-api-access-z2dz8\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:06:51.121820 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:51.121817 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265bd5cb-de44-4eed-bff2-e528cd50c922-maas-api-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:06:51.121820 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:51.121826 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265bd5cb-de44-4eed-bff2-e528cd50c922-openshift-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:06:51.851352 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:51.851324 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" Jul 09 19:06:51.851722 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:51.851343 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72" event={"ID":"265bd5cb-de44-4eed-bff2-e528cd50c922","Type":"ContainerDied","Data":"df50f08fe39d76070ca5538f480033aaaffe471c4a7722942054a6d8a79f31b1"} Jul 09 19:06:51.851722 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:51.851400 2585 scope.go:117] "RemoveContainer" containerID="11cad6f4c62fb9133d7e6043742c3b47b57479884802c594b39f38c4d75c004b" Jul 09 19:06:51.869850 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:51.869823 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72"] Jul 09 19:06:51.874425 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:51.874399 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-231d4590-645c657cb6-nfk72"] Jul 09 19:06:53.842998 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:53.842962 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="265bd5cb-de44-4eed-bff2-e528cd50c922" path="/var/lib/kubelet/pods/265bd5cb-de44-4eed-bff2-e528cd50c922/volumes" Jul 09 19:06:59.096807 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.096778 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj"] Jul 09 19:06:59.097344 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.097124 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="265bd5cb-de44-4eed-bff2-e528cd50c922" containerName="maas-api" Jul 09 19:06:59.097344 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.097135 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="265bd5cb-de44-4eed-bff2-e528cd50c922" containerName="maas-api" Jul 09 19:06:59.097344 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.097220 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="265bd5cb-de44-4eed-bff2-e528cd50c922" containerName="maas-api" Jul 09 19:06:59.100637 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.100614 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:06:59.103273 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.103253 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b10d3200-serving-cert\"" Jul 09 19:06:59.114508 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.114463 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj"] Jul 09 19:06:59.193751 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.193723 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33cdbe56-21a2-4791-8244-b8ada62652d0-maas-api-tls\") pod \"maas-api-e2e-ait-b10d3200-79db769f54-zmprj\" (UID: \"33cdbe56-21a2-4791-8244-b8ada62652d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:06:59.193904 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.193753 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwf99\" (UniqueName: \"kubernetes.io/projected/33cdbe56-21a2-4791-8244-b8ada62652d0-kube-api-access-mwf99\") pod \"maas-api-e2e-ait-b10d3200-79db769f54-zmprj\" (UID: \"33cdbe56-21a2-4791-8244-b8ada62652d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:06:59.193904 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.193836 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33cdbe56-21a2-4791-8244-b8ada62652d0-openshift-service-ca\") pod \"maas-api-e2e-ait-b10d3200-79db769f54-zmprj\" (UID: \"33cdbe56-21a2-4791-8244-b8ada62652d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:06:59.294548 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.294515 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33cdbe56-21a2-4791-8244-b8ada62652d0-maas-api-tls\") pod \"maas-api-e2e-ait-b10d3200-79db769f54-zmprj\" (UID: \"33cdbe56-21a2-4791-8244-b8ada62652d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:06:59.294548 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.294548 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwf99\" (UniqueName: \"kubernetes.io/projected/33cdbe56-21a2-4791-8244-b8ada62652d0-kube-api-access-mwf99\") pod \"maas-api-e2e-ait-b10d3200-79db769f54-zmprj\" (UID: \"33cdbe56-21a2-4791-8244-b8ada62652d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:06:59.294780 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.294572 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33cdbe56-21a2-4791-8244-b8ada62652d0-openshift-service-ca\") pod \"maas-api-e2e-ait-b10d3200-79db769f54-zmprj\" (UID: \"33cdbe56-21a2-4791-8244-b8ada62652d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:06:59.294780 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:06:59.294706 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-serving-cert: secret "maas-api-e2e-ait-b10d3200-serving-cert" not found Jul 09 19:06:59.294861 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:06:59.294784 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33cdbe56-21a2-4791-8244-b8ada62652d0-maas-api-tls podName:33cdbe56-21a2-4791-8244-b8ada62652d0 nodeName:}" failed. No retries permitted until 2026-07-09 19:06:59.794763648 +0000 UTC m=+1536.538746753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/33cdbe56-21a2-4791-8244-b8ada62652d0-maas-api-tls") pod "maas-api-e2e-ait-b10d3200-79db769f54-zmprj" (UID: "33cdbe56-21a2-4791-8244-b8ada62652d0") : secret "maas-api-e2e-ait-b10d3200-serving-cert" not found Jul 09 19:06:59.295231 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.295214 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33cdbe56-21a2-4791-8244-b8ada62652d0-openshift-service-ca\") pod \"maas-api-e2e-ait-b10d3200-79db769f54-zmprj\" (UID: \"33cdbe56-21a2-4791-8244-b8ada62652d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:06:59.305508 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.305476 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwf99\" (UniqueName: \"kubernetes.io/projected/33cdbe56-21a2-4791-8244-b8ada62652d0-kube-api-access-mwf99\") pod \"maas-api-e2e-ait-b10d3200-79db769f54-zmprj\" (UID: \"33cdbe56-21a2-4791-8244-b8ada62652d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:06:59.799264 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.799234 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33cdbe56-21a2-4791-8244-b8ada62652d0-maas-api-tls\") pod \"maas-api-e2e-ait-b10d3200-79db769f54-zmprj\" (UID: \"33cdbe56-21a2-4791-8244-b8ada62652d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:06:59.801889 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:06:59.801863 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33cdbe56-21a2-4791-8244-b8ada62652d0-maas-api-tls\") pod \"maas-api-e2e-ait-b10d3200-79db769f54-zmprj\" (UID: \"33cdbe56-21a2-4791-8244-b8ada62652d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:07:00.011350 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:00.011308 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:07:00.158275 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:00.158251 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj"] Jul 09 19:07:00.160924 ip-10-0-143-175 kubenswrapper[2585]: W0709 19:07:00.160896 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33cdbe56_21a2_4791_8244_b8ada62652d0.slice/crio-92286283cd153779b58c7b9da6bad6bdff155094dfb56ac01d1a087161fdc88e WatchSource:0}: Error finding container 92286283cd153779b58c7b9da6bad6bdff155094dfb56ac01d1a087161fdc88e: Status 404 returned error can't find the container with id 92286283cd153779b58c7b9da6bad6bdff155094dfb56ac01d1a087161fdc88e Jul 09 19:07:00.898193 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:00.898161 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" event={"ID":"33cdbe56-21a2-4791-8244-b8ada62652d0","Type":"ContainerStarted","Data":"8252d972fe1672634ddab5bae6ea74d6dcd2734424260b0789ff64b27064e4b6"} Jul 09 19:07:00.898346 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:00.898209 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" event={"ID":"33cdbe56-21a2-4791-8244-b8ada62652d0","Type":"ContainerStarted","Data":"92286283cd153779b58c7b9da6bad6bdff155094dfb56ac01d1a087161fdc88e"} Jul 09 19:07:00.898346 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:00.898239 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:07:00.918641 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:00.918579 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" podStartSLOduration=1.289640265 podStartE2EDuration="1.918562686s" totalImagesPullingTime="628.922421ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:06:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:07:00.91668189 +0000 UTC m=+1537.660665006" watchObservedRunningTime="2026-07-09 19:07:00.918562686 +0000 UTC m=+1537.662545801" Jul 09 19:07:06.908543 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:06.908514 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:07:06.950909 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:06.950878 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj"] Jul 09 19:07:06.951096 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:06.951073 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" podUID="33cdbe56-21a2-4791-8244-b8ada62652d0" containerName="maas-api" containerID="cri-o://8252d972fe1672634ddab5bae6ea74d6dcd2734424260b0789ff64b27064e4b6" gracePeriod=30 Jul 09 19:07:07.206453 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.206430 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:07:07.270595 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.270559 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwf99\" (UniqueName: \"kubernetes.io/projected/33cdbe56-21a2-4791-8244-b8ada62652d0-kube-api-access-mwf99\") pod \"33cdbe56-21a2-4791-8244-b8ada62652d0\" (UID: \"33cdbe56-21a2-4791-8244-b8ada62652d0\") " Jul 09 19:07:07.270775 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.270640 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33cdbe56-21a2-4791-8244-b8ada62652d0-openshift-service-ca\") pod \"33cdbe56-21a2-4791-8244-b8ada62652d0\" (UID: \"33cdbe56-21a2-4791-8244-b8ada62652d0\") " Jul 09 19:07:07.270775 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.270665 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33cdbe56-21a2-4791-8244-b8ada62652d0-maas-api-tls\") pod \"33cdbe56-21a2-4791-8244-b8ada62652d0\" (UID: \"33cdbe56-21a2-4791-8244-b8ada62652d0\") " Jul 09 19:07:07.271018 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.270994 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33cdbe56-21a2-4791-8244-b8ada62652d0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33cdbe56-21a2-4791-8244-b8ada62652d0" (UID: "33cdbe56-21a2-4791-8244-b8ada62652d0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:07:07.272980 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.272954 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33cdbe56-21a2-4791-8244-b8ada62652d0-kube-api-access-mwf99" (OuterVolumeSpecName: "kube-api-access-mwf99") pod "33cdbe56-21a2-4791-8244-b8ada62652d0" (UID: "33cdbe56-21a2-4791-8244-b8ada62652d0"). InnerVolumeSpecName "kube-api-access-mwf99". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:07:07.272980 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.272960 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33cdbe56-21a2-4791-8244-b8ada62652d0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33cdbe56-21a2-4791-8244-b8ada62652d0" (UID: "33cdbe56-21a2-4791-8244-b8ada62652d0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:07:07.371298 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.371254 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwf99\" (UniqueName: \"kubernetes.io/projected/33cdbe56-21a2-4791-8244-b8ada62652d0-kube-api-access-mwf99\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:07:07.371298 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.371290 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33cdbe56-21a2-4791-8244-b8ada62652d0-openshift-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:07:07.371298 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.371301 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33cdbe56-21a2-4791-8244-b8ada62652d0-maas-api-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:07:07.934203 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.934170 2585 generic.go:356] "Generic (PLEG): container finished" podID="33cdbe56-21a2-4791-8244-b8ada62652d0" containerID="8252d972fe1672634ddab5bae6ea74d6dcd2734424260b0789ff64b27064e4b6" exitCode=0 Jul 09 19:07:07.934647 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.934220 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" Jul 09 19:07:07.934647 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.934251 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" event={"ID":"33cdbe56-21a2-4791-8244-b8ada62652d0","Type":"ContainerDied","Data":"8252d972fe1672634ddab5bae6ea74d6dcd2734424260b0789ff64b27064e4b6"} Jul 09 19:07:07.934647 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.934306 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj" event={"ID":"33cdbe56-21a2-4791-8244-b8ada62652d0","Type":"ContainerDied","Data":"92286283cd153779b58c7b9da6bad6bdff155094dfb56ac01d1a087161fdc88e"} Jul 09 19:07:07.934647 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.934329 2585 scope.go:117] "RemoveContainer" containerID="8252d972fe1672634ddab5bae6ea74d6dcd2734424260b0789ff64b27064e4b6" Jul 09 19:07:07.944290 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.944256 2585 scope.go:117] "RemoveContainer" containerID="8252d972fe1672634ddab5bae6ea74d6dcd2734424260b0789ff64b27064e4b6" Jul 09 19:07:07.944740 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:07:07.944705 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8252d972fe1672634ddab5bae6ea74d6dcd2734424260b0789ff64b27064e4b6\": container with ID starting with 8252d972fe1672634ddab5bae6ea74d6dcd2734424260b0789ff64b27064e4b6 not found: ID does not exist" containerID="8252d972fe1672634ddab5bae6ea74d6dcd2734424260b0789ff64b27064e4b6" Jul 09 19:07:07.944847 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.944760 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8252d972fe1672634ddab5bae6ea74d6dcd2734424260b0789ff64b27064e4b6"} err="failed to get container status \"8252d972fe1672634ddab5bae6ea74d6dcd2734424260b0789ff64b27064e4b6\": rpc error: code = NotFound desc = could not find container \"8252d972fe1672634ddab5bae6ea74d6dcd2734424260b0789ff64b27064e4b6\": container with ID starting with 8252d972fe1672634ddab5bae6ea74d6dcd2734424260b0789ff64b27064e4b6 not found: ID does not exist" Jul 09 19:07:07.961167 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.961140 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj"] Jul 09 19:07:07.970926 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:07.970904 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b10d3200-79db769f54-zmprj"] Jul 09 19:07:09.842688 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:09.842652 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33cdbe56-21a2-4791-8244-b8ada62652d0" path="/var/lib/kubelet/pods/33cdbe56-21a2-4791-8244-b8ada62652d0/volumes" Jul 09 19:07:16.167202 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.167165 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg"] Jul 09 19:07:16.167572 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.167552 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33cdbe56-21a2-4791-8244-b8ada62652d0" containerName="maas-api" Jul 09 19:07:16.167572 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.167564 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="33cdbe56-21a2-4791-8244-b8ada62652d0" containerName="maas-api" Jul 09 19:07:16.167666 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.167654 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="33cdbe56-21a2-4791-8244-b8ada62652d0" containerName="maas-api" Jul 09 19:07:16.172137 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.172114 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:16.181991 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.181968 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-a9f3f7be-serving-cert\"" Jul 09 19:07:16.185492 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.185469 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg"] Jul 09 19:07:16.260112 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.260062 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b86dh\" (UniqueName: \"kubernetes.io/projected/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-kube-api-access-b86dh\") pod \"maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg\" (UID: \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:16.260302 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.260136 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-maas-api-tls\") pod \"maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg\" (UID: \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:16.260302 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.260165 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-openshift-service-ca\") pod \"maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg\" (UID: \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:16.361376 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.361331 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b86dh\" (UniqueName: \"kubernetes.io/projected/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-kube-api-access-b86dh\") pod \"maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg\" (UID: \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:16.361376 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.361386 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-maas-api-tls\") pod \"maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg\" (UID: \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:16.361646 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.361410 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-openshift-service-ca\") pod \"maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg\" (UID: \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:16.361646 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:07:16.361509 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-serving-cert: secret "maas-api-e2e-derive-a9f3f7be-serving-cert" not found Jul 09 19:07:16.361646 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:07:16.361616 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-maas-api-tls podName:eb114429-e5a6-4bc2-b1fe-c1f31e3377b9 nodeName:}" failed. No retries permitted until 2026-07-09 19:07:16.861564799 +0000 UTC m=+1553.605547910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-maas-api-tls") pod "maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" (UID: "eb114429-e5a6-4bc2-b1fe-c1f31e3377b9") : secret "maas-api-e2e-derive-a9f3f7be-serving-cert" not found Jul 09 19:07:16.362144 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.362125 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-openshift-service-ca\") pod \"maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg\" (UID: \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:16.388440 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.388410 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b86dh\" (UniqueName: \"kubernetes.io/projected/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-kube-api-access-b86dh\") pod \"maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg\" (UID: \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:16.866174 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.866144 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-maas-api-tls\") pod \"maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg\" (UID: \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:16.868848 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:16.868825 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-maas-api-tls\") pod \"maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg\" (UID: \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:17.083022 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:17.082993 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:17.297775 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:17.297733 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg"] Jul 09 19:07:17.301095 ip-10-0-143-175 kubenswrapper[2585]: W0709 19:07:17.301061 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb114429_e5a6_4bc2_b1fe_c1f31e3377b9.slice/crio-7a0092eceaa30c7d520235fd8b4a872095e489f24ee0a4a3038eaec0f5d06515 WatchSource:0}: Error finding container 7a0092eceaa30c7d520235fd8b4a872095e489f24ee0a4a3038eaec0f5d06515: Status 404 returned error can't find the container with id 7a0092eceaa30c7d520235fd8b4a872095e489f24ee0a4a3038eaec0f5d06515 Jul 09 19:07:17.982999 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:17.982963 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" event={"ID":"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9","Type":"ContainerStarted","Data":"8cc1f381c8e01806522001ad8e6f7d8b45da90ac17fe288c89325c40e2569314"} Jul 09 19:07:17.983199 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:17.983011 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" event={"ID":"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9","Type":"ContainerStarted","Data":"7a0092eceaa30c7d520235fd8b4a872095e489f24ee0a4a3038eaec0f5d06515"} Jul 09 19:07:17.983199 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:17.983054 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:18.019716 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:18.019567 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" podStartSLOduration=1.584169552 podStartE2EDuration="2.019546425s" totalImagesPullingTime="435.376873ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:07:18.018755618 +0000 UTC m=+1554.762738766" watchObservedRunningTime="2026-07-09 19:07:18.019546425 +0000 UTC m=+1554.763529540" Jul 09 19:07:23.226194 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:23.226159 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg"] Jul 09 19:07:23.226654 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:23.226428 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" podUID="eb114429-e5a6-4bc2-b1fe-c1f31e3377b9" containerName="maas-api" containerID="cri-o://8cc1f381c8e01806522001ad8e6f7d8b45da90ac17fe288c89325c40e2569314" gracePeriod=30 Jul 09 19:07:23.232213 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:23.232183 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:23.500033 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:23.500007 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:23.528044 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:23.528014 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-maas-api-tls\") pod \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\" (UID: \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\") " Jul 09 19:07:23.528239 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:23.528072 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-openshift-service-ca\") pod \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\" (UID: \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\") " Jul 09 19:07:23.528310 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:23.528244 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b86dh\" (UniqueName: \"kubernetes.io/projected/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-kube-api-access-b86dh\") pod \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\" (UID: \"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9\") " Jul 09 19:07:23.528445 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:23.528420 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eb114429-e5a6-4bc2-b1fe-c1f31e3377b9" (UID: "eb114429-e5a6-4bc2-b1fe-c1f31e3377b9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:07:23.528612 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:23.528572 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-openshift-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:07:23.530216 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:23.530192 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eb114429-e5a6-4bc2-b1fe-c1f31e3377b9" (UID: "eb114429-e5a6-4bc2-b1fe-c1f31e3377b9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:07:23.530329 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:23.530308 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-kube-api-access-b86dh" (OuterVolumeSpecName: "kube-api-access-b86dh") pod "eb114429-e5a6-4bc2-b1fe-c1f31e3377b9" (UID: "eb114429-e5a6-4bc2-b1fe-c1f31e3377b9"). InnerVolumeSpecName "kube-api-access-b86dh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:07:23.629395 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:23.629349 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b86dh\" (UniqueName: \"kubernetes.io/projected/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-kube-api-access-b86dh\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:07:23.629395 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:23.629386 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9-maas-api-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:07:24.011067 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:24.011030 2585 generic.go:356] "Generic (PLEG): container finished" podID="eb114429-e5a6-4bc2-b1fe-c1f31e3377b9" containerID="8cc1f381c8e01806522001ad8e6f7d8b45da90ac17fe288c89325c40e2569314" exitCode=0 Jul 09 19:07:24.011258 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:24.011074 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" Jul 09 19:07:24.011258 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:24.011076 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" event={"ID":"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9","Type":"ContainerDied","Data":"8cc1f381c8e01806522001ad8e6f7d8b45da90ac17fe288c89325c40e2569314"} Jul 09 19:07:24.011258 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:24.011210 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg" event={"ID":"eb114429-e5a6-4bc2-b1fe-c1f31e3377b9","Type":"ContainerDied","Data":"7a0092eceaa30c7d520235fd8b4a872095e489f24ee0a4a3038eaec0f5d06515"} Jul 09 19:07:24.011258 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:24.011247 2585 scope.go:117] "RemoveContainer" containerID="8cc1f381c8e01806522001ad8e6f7d8b45da90ac17fe288c89325c40e2569314" Jul 09 19:07:24.019995 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:24.019975 2585 scope.go:117] "RemoveContainer" containerID="8cc1f381c8e01806522001ad8e6f7d8b45da90ac17fe288c89325c40e2569314" Jul 09 19:07:24.020278 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:07:24.020254 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cc1f381c8e01806522001ad8e6f7d8b45da90ac17fe288c89325c40e2569314\": container with ID starting with 8cc1f381c8e01806522001ad8e6f7d8b45da90ac17fe288c89325c40e2569314 not found: ID does not exist" containerID="8cc1f381c8e01806522001ad8e6f7d8b45da90ac17fe288c89325c40e2569314" Jul 09 19:07:24.020321 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:24.020292 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc1f381c8e01806522001ad8e6f7d8b45da90ac17fe288c89325c40e2569314"} err="failed to get container status \"8cc1f381c8e01806522001ad8e6f7d8b45da90ac17fe288c89325c40e2569314\": rpc error: code = NotFound desc = could not find container \"8cc1f381c8e01806522001ad8e6f7d8b45da90ac17fe288c89325c40e2569314\": container with ID starting with 8cc1f381c8e01806522001ad8e6f7d8b45da90ac17fe288c89325c40e2569314 not found: ID does not exist" Jul 09 19:07:24.031203 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:24.031174 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg"] Jul 09 19:07:24.037841 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:24.037817 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a9f3f7be-6c47885f5b-4npzg"] Jul 09 19:07:25.849343 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:07:25.849306 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb114429-e5a6-4bc2-b1fe-c1f31e3377b9" path="/var/lib/kubelet/pods/eb114429-e5a6-4bc2-b1fe-c1f31e3377b9/volumes" Jul 09 19:11:00.799656 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.799616 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m"] Jul 09 19:11:00.800246 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.800134 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb114429-e5a6-4bc2-b1fe-c1f31e3377b9" containerName="maas-api" Jul 09 19:11:00.800246 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.800154 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb114429-e5a6-4bc2-b1fe-c1f31e3377b9" containerName="maas-api" Jul 09 19:11:00.800371 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.800265 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb114429-e5a6-4bc2-b1fe-c1f31e3377b9" containerName="maas-api" Jul 09 19:11:00.803250 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.803229 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" Jul 09 19:11:00.806018 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.805994 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-98995b74-serving-cert\"" Jul 09 19:11:00.814042 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.814017 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m"] Jul 09 19:11:00.873693 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.873660 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjh6n\" (UniqueName: \"kubernetes.io/projected/13b96da9-252e-4159-a2f5-da68db479fcb-kube-api-access-gjh6n\") pod \"maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m\" (UID: \"13b96da9-252e-4159-a2f5-da68db479fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" Jul 09 19:11:00.873884 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.873713 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b96da9-252e-4159-a2f5-da68db479fcb-maas-api-tls\") pod \"maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m\" (UID: \"13b96da9-252e-4159-a2f5-da68db479fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" Jul 09 19:11:00.873884 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.873788 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13b96da9-252e-4159-a2f5-da68db479fcb-openshift-service-ca\") pod \"maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m\" (UID: \"13b96da9-252e-4159-a2f5-da68db479fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" Jul 09 19:11:00.974463 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.974426 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13b96da9-252e-4159-a2f5-da68db479fcb-openshift-service-ca\") pod \"maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m\" (UID: \"13b96da9-252e-4159-a2f5-da68db479fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" Jul 09 19:11:00.974681 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.974480 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjh6n\" (UniqueName: \"kubernetes.io/projected/13b96da9-252e-4159-a2f5-da68db479fcb-kube-api-access-gjh6n\") pod \"maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m\" (UID: \"13b96da9-252e-4159-a2f5-da68db479fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" Jul 09 19:11:00.974681 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.974507 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b96da9-252e-4159-a2f5-da68db479fcb-maas-api-tls\") pod \"maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m\" (UID: \"13b96da9-252e-4159-a2f5-da68db479fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" Jul 09 19:11:00.975146 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.975121 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13b96da9-252e-4159-a2f5-da68db479fcb-openshift-service-ca\") pod \"maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m\" (UID: \"13b96da9-252e-4159-a2f5-da68db479fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" Jul 09 19:11:00.977199 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.977170 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b96da9-252e-4159-a2f5-da68db479fcb-maas-api-tls\") pod \"maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m\" (UID: \"13b96da9-252e-4159-a2f5-da68db479fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" Jul 09 19:11:00.984342 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:00.984313 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjh6n\" (UniqueName: \"kubernetes.io/projected/13b96da9-252e-4159-a2f5-da68db479fcb-kube-api-access-gjh6n\") pod \"maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m\" (UID: \"13b96da9-252e-4159-a2f5-da68db479fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" Jul 09 19:11:01.115688 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:01.115572 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" Jul 09 19:11:01.276856 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:01.276813 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m"] Jul 09 19:11:01.279648 ip-10-0-143-175 kubenswrapper[2585]: W0709 19:11:01.279618 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13b96da9_252e_4159_a2f5_da68db479fcb.slice/crio-189ebdbc47776e0383b13627adddc300056c3fa2c9773c4cd7683dbf832d588d WatchSource:0}: Error finding container 189ebdbc47776e0383b13627adddc300056c3fa2c9773c4cd7683dbf832d588d: Status 404 returned error can't find the container with id 189ebdbc47776e0383b13627adddc300056c3fa2c9773c4cd7683dbf832d588d Jul 09 19:11:02.022378 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:02.022342 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" event={"ID":"13b96da9-252e-4159-a2f5-da68db479fcb","Type":"ContainerStarted","Data":"858294ae3ed6790b456b88112d76db2497d5b6a6d6ef803446f9598b833bef08"} Jul 09 19:11:02.022856 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:02.022391 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" event={"ID":"13b96da9-252e-4159-a2f5-da68db479fcb","Type":"ContainerStarted","Data":"189ebdbc47776e0383b13627adddc300056c3fa2c9773c4cd7683dbf832d588d"} Jul 09 19:11:02.022856 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:02.022474 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" Jul 09 19:11:02.045990 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:02.045928 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" podStartSLOduration=1.4279599649999999 podStartE2EDuration="2.045906595s" totalImagesPullingTime="617.94663ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:11:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:11:02.044252078 +0000 UTC m=+1778.788235195" watchObservedRunningTime="2026-07-09 19:11:02.045906595 +0000 UTC m=+1778.789889710" Jul 09 19:11:08.033501 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:08.033473 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" Jul 09 19:11:13.822994 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:13.822956 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m"] Jul 09 19:11:13.823468 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:13.823177 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" podUID="13b96da9-252e-4159-a2f5-da68db479fcb" containerName="maas-api" containerID="cri-o://858294ae3ed6790b456b88112d76db2497d5b6a6d6ef803446f9598b833bef08" gracePeriod=30 Jul 09 19:11:14.071397 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.071367 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" Jul 09 19:11:14.089842 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.089751 2585 generic.go:356] "Generic (PLEG): container finished" podID="13b96da9-252e-4159-a2f5-da68db479fcb" containerID="858294ae3ed6790b456b88112d76db2497d5b6a6d6ef803446f9598b833bef08" exitCode=0 Jul 09 19:11:14.089842 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.089797 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" Jul 09 19:11:14.090029 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.089841 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" event={"ID":"13b96da9-252e-4159-a2f5-da68db479fcb","Type":"ContainerDied","Data":"858294ae3ed6790b456b88112d76db2497d5b6a6d6ef803446f9598b833bef08"} Jul 09 19:11:14.090029 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.089895 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m" event={"ID":"13b96da9-252e-4159-a2f5-da68db479fcb","Type":"ContainerDied","Data":"189ebdbc47776e0383b13627adddc300056c3fa2c9773c4cd7683dbf832d588d"} Jul 09 19:11:14.090029 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.089925 2585 scope.go:117] "RemoveContainer" containerID="858294ae3ed6790b456b88112d76db2497d5b6a6d6ef803446f9598b833bef08" Jul 09 19:11:14.099346 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.099324 2585 scope.go:117] "RemoveContainer" containerID="858294ae3ed6790b456b88112d76db2497d5b6a6d6ef803446f9598b833bef08" Jul 09 19:11:14.099659 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:11:14.099639 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"858294ae3ed6790b456b88112d76db2497d5b6a6d6ef803446f9598b833bef08\": container with ID starting with 858294ae3ed6790b456b88112d76db2497d5b6a6d6ef803446f9598b833bef08 not found: ID does not exist" containerID="858294ae3ed6790b456b88112d76db2497d5b6a6d6ef803446f9598b833bef08" Jul 09 19:11:14.099714 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.099668 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858294ae3ed6790b456b88112d76db2497d5b6a6d6ef803446f9598b833bef08"} err="failed to get container status \"858294ae3ed6790b456b88112d76db2497d5b6a6d6ef803446f9598b833bef08\": rpc error: code = NotFound desc = could not find container \"858294ae3ed6790b456b88112d76db2497d5b6a6d6ef803446f9598b833bef08\": container with ID starting with 858294ae3ed6790b456b88112d76db2497d5b6a6d6ef803446f9598b833bef08 not found: ID does not exist" Jul 09 19:11:14.192823 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.192794 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b96da9-252e-4159-a2f5-da68db479fcb-maas-api-tls\") pod \"13b96da9-252e-4159-a2f5-da68db479fcb\" (UID: \"13b96da9-252e-4159-a2f5-da68db479fcb\") " Jul 09 19:11:14.192823 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.192829 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13b96da9-252e-4159-a2f5-da68db479fcb-openshift-service-ca\") pod \"13b96da9-252e-4159-a2f5-da68db479fcb\" (UID: \"13b96da9-252e-4159-a2f5-da68db479fcb\") " Jul 09 19:11:14.193065 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.192850 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjh6n\" (UniqueName: \"kubernetes.io/projected/13b96da9-252e-4159-a2f5-da68db479fcb-kube-api-access-gjh6n\") pod \"13b96da9-252e-4159-a2f5-da68db479fcb\" (UID: \"13b96da9-252e-4159-a2f5-da68db479fcb\") " Jul 09 19:11:14.193236 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.193208 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13b96da9-252e-4159-a2f5-da68db479fcb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "13b96da9-252e-4159-a2f5-da68db479fcb" (UID: "13b96da9-252e-4159-a2f5-da68db479fcb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:11:14.194965 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.194934 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13b96da9-252e-4159-a2f5-da68db479fcb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "13b96da9-252e-4159-a2f5-da68db479fcb" (UID: "13b96da9-252e-4159-a2f5-da68db479fcb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:11:14.195037 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.195018 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13b96da9-252e-4159-a2f5-da68db479fcb-kube-api-access-gjh6n" (OuterVolumeSpecName: "kube-api-access-gjh6n") pod "13b96da9-252e-4159-a2f5-da68db479fcb" (UID: "13b96da9-252e-4159-a2f5-da68db479fcb"). InnerVolumeSpecName "kube-api-access-gjh6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:11:14.294304 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.294277 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b96da9-252e-4159-a2f5-da68db479fcb-maas-api-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:11:14.294304 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.294303 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13b96da9-252e-4159-a2f5-da68db479fcb-openshift-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:11:14.294304 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.294313 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjh6n\" (UniqueName: \"kubernetes.io/projected/13b96da9-252e-4159-a2f5-da68db479fcb-kube-api-access-gjh6n\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:11:14.412498 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.412461 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m"] Jul 09 19:11:14.417357 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:14.417322 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-98995b74-54d754d7cd-p6c5m"] Jul 09 19:11:15.842558 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:15.842524 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13b96da9-252e-4159-a2f5-da68db479fcb" path="/var/lib/kubelet/pods/13b96da9-252e-4159-a2f5-da68db479fcb/volumes" Jul 09 19:11:24.514255 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:24.514131 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 19:11:24.534537 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:24.516489 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 19:11:44.851679 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:44.851639 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s"] Jul 09 19:11:44.852218 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:44.852198 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b96da9-252e-4159-a2f5-da68db479fcb" containerName="maas-api" Jul 09 19:11:44.852270 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:44.852224 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b96da9-252e-4159-a2f5-da68db479fcb" containerName="maas-api" Jul 09 19:11:44.852355 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:44.852343 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="13b96da9-252e-4159-a2f5-da68db479fcb" containerName="maas-api" Jul 09 19:11:44.855278 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:44.855261 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" Jul 09 19:11:44.857879 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:44.857850 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9cef6b5e-serving-cert\"" Jul 09 19:11:44.865444 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:44.865416 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s"] Jul 09 19:11:44.970138 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:44.970099 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37bd6615-4691-49c8-b289-0b8d99d1b0b2-openshift-service-ca\") pod \"maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s\" (UID: \"37bd6615-4691-49c8-b289-0b8d99d1b0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" Jul 09 19:11:44.970330 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:44.970151 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v7lh\" (UniqueName: \"kubernetes.io/projected/37bd6615-4691-49c8-b289-0b8d99d1b0b2-kube-api-access-6v7lh\") pod \"maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s\" (UID: \"37bd6615-4691-49c8-b289-0b8d99d1b0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" Jul 09 19:11:44.970330 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:44.970175 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37bd6615-4691-49c8-b289-0b8d99d1b0b2-maas-api-tls\") pod \"maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s\" (UID: \"37bd6615-4691-49c8-b289-0b8d99d1b0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" Jul 09 19:11:45.071184 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:45.071151 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37bd6615-4691-49c8-b289-0b8d99d1b0b2-openshift-service-ca\") pod \"maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s\" (UID: \"37bd6615-4691-49c8-b289-0b8d99d1b0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" Jul 09 19:11:45.071366 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:45.071195 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v7lh\" (UniqueName: \"kubernetes.io/projected/37bd6615-4691-49c8-b289-0b8d99d1b0b2-kube-api-access-6v7lh\") pod \"maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s\" (UID: \"37bd6615-4691-49c8-b289-0b8d99d1b0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" Jul 09 19:11:45.071366 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:45.071220 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37bd6615-4691-49c8-b289-0b8d99d1b0b2-maas-api-tls\") pod \"maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s\" (UID: \"37bd6615-4691-49c8-b289-0b8d99d1b0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" Jul 09 19:11:45.071812 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:45.071790 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37bd6615-4691-49c8-b289-0b8d99d1b0b2-openshift-service-ca\") pod \"maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s\" (UID: \"37bd6615-4691-49c8-b289-0b8d99d1b0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" Jul 09 19:11:45.073815 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:45.073795 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37bd6615-4691-49c8-b289-0b8d99d1b0b2-maas-api-tls\") pod \"maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s\" (UID: \"37bd6615-4691-49c8-b289-0b8d99d1b0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" Jul 09 19:11:45.079954 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:45.079930 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v7lh\" (UniqueName: \"kubernetes.io/projected/37bd6615-4691-49c8-b289-0b8d99d1b0b2-kube-api-access-6v7lh\") pod \"maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s\" (UID: \"37bd6615-4691-49c8-b289-0b8d99d1b0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" Jul 09 19:11:45.167753 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:45.167660 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" Jul 09 19:11:45.311111 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:45.311079 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s"] Jul 09 19:11:45.314542 ip-10-0-143-175 kubenswrapper[2585]: W0709 19:11:45.314511 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37bd6615_4691_49c8_b289_0b8d99d1b0b2.slice/crio-9beb47d0d15acd5994304fac2a44c9e6f7a7477b8952a65c0d241a00104c9bcf WatchSource:0}: Error finding container 9beb47d0d15acd5994304fac2a44c9e6f7a7477b8952a65c0d241a00104c9bcf: Status 404 returned error can't find the container with id 9beb47d0d15acd5994304fac2a44c9e6f7a7477b8952a65c0d241a00104c9bcf Jul 09 19:11:45.316032 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:45.316012 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 19:11:46.238799 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:46.238763 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" event={"ID":"37bd6615-4691-49c8-b289-0b8d99d1b0b2","Type":"ContainerStarted","Data":"b7295c205cb109313b408496a2fe5baf03a00d7d4d47be2140065bc7fb809943"} Jul 09 19:11:46.239210 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:46.238815 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" event={"ID":"37bd6615-4691-49c8-b289-0b8d99d1b0b2","Type":"ContainerStarted","Data":"9beb47d0d15acd5994304fac2a44c9e6f7a7477b8952a65c0d241a00104c9bcf"} Jul 09 19:11:46.239210 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:46.238874 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" Jul 09 19:11:46.271874 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:46.271816 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" podStartSLOduration=1.86229446 podStartE2EDuration="2.271799725s" totalImagesPullingTime="409.505265ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:11:46.269384894 +0000 UTC m=+1823.013368020" watchObservedRunningTime="2026-07-09 19:11:46.271799725 +0000 UTC m=+1823.015782839" Jul 09 19:11:52.249931 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:11:52.249905 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" Jul 09 19:12:17.754404 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:17.754360 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s"] Jul 09 19:12:17.754992 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:17.754664 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" podUID="37bd6615-4691-49c8-b289-0b8d99d1b0b2" containerName="maas-api" containerID="cri-o://b7295c205cb109313b408496a2fe5baf03a00d7d4d47be2140065bc7fb809943" gracePeriod=30 Jul 09 19:12:18.004545 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.004482 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" Jul 09 19:12:18.069862 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.069824 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v7lh\" (UniqueName: \"kubernetes.io/projected/37bd6615-4691-49c8-b289-0b8d99d1b0b2-kube-api-access-6v7lh\") pod \"37bd6615-4691-49c8-b289-0b8d99d1b0b2\" (UID: \"37bd6615-4691-49c8-b289-0b8d99d1b0b2\") " Jul 09 19:12:18.070016 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.069895 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37bd6615-4691-49c8-b289-0b8d99d1b0b2-maas-api-tls\") pod \"37bd6615-4691-49c8-b289-0b8d99d1b0b2\" (UID: \"37bd6615-4691-49c8-b289-0b8d99d1b0b2\") " Jul 09 19:12:18.070016 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.069946 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37bd6615-4691-49c8-b289-0b8d99d1b0b2-openshift-service-ca\") pod \"37bd6615-4691-49c8-b289-0b8d99d1b0b2\" (UID: \"37bd6615-4691-49c8-b289-0b8d99d1b0b2\") " Jul 09 19:12:18.070366 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.070340 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37bd6615-4691-49c8-b289-0b8d99d1b0b2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "37bd6615-4691-49c8-b289-0b8d99d1b0b2" (UID: "37bd6615-4691-49c8-b289-0b8d99d1b0b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:12:18.072229 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.072201 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37bd6615-4691-49c8-b289-0b8d99d1b0b2-kube-api-access-6v7lh" (OuterVolumeSpecName: "kube-api-access-6v7lh") pod "37bd6615-4691-49c8-b289-0b8d99d1b0b2" (UID: "37bd6615-4691-49c8-b289-0b8d99d1b0b2"). InnerVolumeSpecName "kube-api-access-6v7lh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:12:18.072346 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.072283 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37bd6615-4691-49c8-b289-0b8d99d1b0b2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "37bd6615-4691-49c8-b289-0b8d99d1b0b2" (UID: "37bd6615-4691-49c8-b289-0b8d99d1b0b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:12:18.171319 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.171290 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37bd6615-4691-49c8-b289-0b8d99d1b0b2-openshift-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:12:18.171319 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.171314 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v7lh\" (UniqueName: \"kubernetes.io/projected/37bd6615-4691-49c8-b289-0b8d99d1b0b2-kube-api-access-6v7lh\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:12:18.171319 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.171325 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37bd6615-4691-49c8-b289-0b8d99d1b0b2-maas-api-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:12:18.398006 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.397914 2585 generic.go:356] "Generic (PLEG): container finished" podID="37bd6615-4691-49c8-b289-0b8d99d1b0b2" containerID="b7295c205cb109313b408496a2fe5baf03a00d7d4d47be2140065bc7fb809943" exitCode=0 Jul 09 19:12:18.398006 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.397966 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" Jul 09 19:12:18.398221 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.398003 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" event={"ID":"37bd6615-4691-49c8-b289-0b8d99d1b0b2","Type":"ContainerDied","Data":"b7295c205cb109313b408496a2fe5baf03a00d7d4d47be2140065bc7fb809943"} Jul 09 19:12:18.398221 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.398059 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s" event={"ID":"37bd6615-4691-49c8-b289-0b8d99d1b0b2","Type":"ContainerDied","Data":"9beb47d0d15acd5994304fac2a44c9e6f7a7477b8952a65c0d241a00104c9bcf"} Jul 09 19:12:18.398221 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.398082 2585 scope.go:117] "RemoveContainer" containerID="b7295c205cb109313b408496a2fe5baf03a00d7d4d47be2140065bc7fb809943" Jul 09 19:12:18.407295 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.407281 2585 scope.go:117] "RemoveContainer" containerID="b7295c205cb109313b408496a2fe5baf03a00d7d4d47be2140065bc7fb809943" Jul 09 19:12:18.407570 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:12:18.407549 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7295c205cb109313b408496a2fe5baf03a00d7d4d47be2140065bc7fb809943\": container with ID starting with b7295c205cb109313b408496a2fe5baf03a00d7d4d47be2140065bc7fb809943 not found: ID does not exist" containerID="b7295c205cb109313b408496a2fe5baf03a00d7d4d47be2140065bc7fb809943" Jul 09 19:12:18.407705 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.407686 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7295c205cb109313b408496a2fe5baf03a00d7d4d47be2140065bc7fb809943"} err="failed to get container status \"b7295c205cb109313b408496a2fe5baf03a00d7d4d47be2140065bc7fb809943\": rpc error: code = NotFound desc = could not find container \"b7295c205cb109313b408496a2fe5baf03a00d7d4d47be2140065bc7fb809943\": container with ID starting with b7295c205cb109313b408496a2fe5baf03a00d7d4d47be2140065bc7fb809943 not found: ID does not exist" Jul 09 19:12:18.429255 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.429224 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s"] Jul 09 19:12:18.433424 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:18.433400 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9cef6b5e-5df9fcf79b-hft9s"] Jul 09 19:12:19.842741 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:12:19.842706 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37bd6615-4691-49c8-b289-0b8d99d1b0b2" path="/var/lib/kubelet/pods/37bd6615-4691-49c8-b289-0b8d99d1b0b2/volumes" Jul 09 19:15:33.283086 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.283052 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs"] Jul 09 19:15:33.284143 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.284121 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37bd6615-4691-49c8-b289-0b8d99d1b0b2" containerName="maas-api" Jul 09 19:15:33.284143 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.284145 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="37bd6615-4691-49c8-b289-0b8d99d1b0b2" containerName="maas-api" Jul 09 19:15:33.284276 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.284212 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="37bd6615-4691-49c8-b289-0b8d99d1b0b2" containerName="maas-api" Jul 09 19:15:33.287463 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.287443 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" Jul 09 19:15:33.290476 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.290456 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-575766-serving-cert\"" Jul 09 19:15:33.297223 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.297157 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs"] Jul 09 19:15:33.403383 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.403351 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fgr4\" (UniqueName: \"kubernetes.io/projected/d22cafa1-ce03-411e-a3c4-314fcb70178f-kube-api-access-2fgr4\") pod \"maas-api-e2e-inf-a-575766-6556c67765-zqgjs\" (UID: \"d22cafa1-ce03-411e-a3c4-314fcb70178f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" Jul 09 19:15:33.403564 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.403412 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d22cafa1-ce03-411e-a3c4-314fcb70178f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-575766-6556c67765-zqgjs\" (UID: \"d22cafa1-ce03-411e-a3c4-314fcb70178f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" Jul 09 19:15:33.403564 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.403544 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d22cafa1-ce03-411e-a3c4-314fcb70178f-maas-api-tls\") pod \"maas-api-e2e-inf-a-575766-6556c67765-zqgjs\" (UID: \"d22cafa1-ce03-411e-a3c4-314fcb70178f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" Jul 09 19:15:33.504931 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.504889 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d22cafa1-ce03-411e-a3c4-314fcb70178f-maas-api-tls\") pod \"maas-api-e2e-inf-a-575766-6556c67765-zqgjs\" (UID: \"d22cafa1-ce03-411e-a3c4-314fcb70178f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" Jul 09 19:15:33.505142 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.504952 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fgr4\" (UniqueName: \"kubernetes.io/projected/d22cafa1-ce03-411e-a3c4-314fcb70178f-kube-api-access-2fgr4\") pod \"maas-api-e2e-inf-a-575766-6556c67765-zqgjs\" (UID: \"d22cafa1-ce03-411e-a3c4-314fcb70178f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" Jul 09 19:15:33.505142 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.505003 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d22cafa1-ce03-411e-a3c4-314fcb70178f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-575766-6556c67765-zqgjs\" (UID: \"d22cafa1-ce03-411e-a3c4-314fcb70178f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" Jul 09 19:15:33.505721 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.505700 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d22cafa1-ce03-411e-a3c4-314fcb70178f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-575766-6556c67765-zqgjs\" (UID: \"d22cafa1-ce03-411e-a3c4-314fcb70178f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" Jul 09 19:15:33.507461 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.507442 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d22cafa1-ce03-411e-a3c4-314fcb70178f-maas-api-tls\") pod \"maas-api-e2e-inf-a-575766-6556c67765-zqgjs\" (UID: \"d22cafa1-ce03-411e-a3c4-314fcb70178f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" Jul 09 19:15:33.514749 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.514726 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fgr4\" (UniqueName: \"kubernetes.io/projected/d22cafa1-ce03-411e-a3c4-314fcb70178f-kube-api-access-2fgr4\") pod \"maas-api-e2e-inf-a-575766-6556c67765-zqgjs\" (UID: \"d22cafa1-ce03-411e-a3c4-314fcb70178f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" Jul 09 19:15:33.598633 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.598523 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" Jul 09 19:15:33.743525 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:33.743504 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs"] Jul 09 19:15:33.746235 ip-10-0-143-175 kubenswrapper[2585]: W0709 19:15:33.746205 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd22cafa1_ce03_411e_a3c4_314fcb70178f.slice/crio-e5ddf483decc86137477ae27b17502797c6ba866d316903244058d4596bf723a WatchSource:0}: Error finding container e5ddf483decc86137477ae27b17502797c6ba866d316903244058d4596bf723a: Status 404 returned error can't find the container with id e5ddf483decc86137477ae27b17502797c6ba866d316903244058d4596bf723a Jul 09 19:15:34.293516 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:34.293478 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" event={"ID":"d22cafa1-ce03-411e-a3c4-314fcb70178f","Type":"ContainerStarted","Data":"e5ddf483decc86137477ae27b17502797c6ba866d316903244058d4596bf723a"} Jul 09 19:15:35.299776 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:35.299736 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" event={"ID":"d22cafa1-ce03-411e-a3c4-314fcb70178f","Type":"ContainerStarted","Data":"df909ff3d1303b144efba584faef7368db02974cce7502b5c7b1b2abc916d98a"} Jul 09 19:15:35.300222 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:35.299841 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" Jul 09 19:15:35.319620 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:35.319559 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" podStartSLOduration=1.717061371 podStartE2EDuration="2.31954405s" totalImagesPullingTime="602.482679ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:15:35.317829905 +0000 UTC m=+2052.061813030" watchObservedRunningTime="2026-07-09 19:15:35.31954405 +0000 UTC m=+2052.063527164" Jul 09 19:15:41.139420 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.139386 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86"] Jul 09 19:15:41.146079 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.146045 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" Jul 09 19:15:41.150495 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.149946 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-8d3645-serving-cert\"" Jul 09 19:15:41.155567 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.155540 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86"] Jul 09 19:15:41.275015 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.274976 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlskv\" (UniqueName: \"kubernetes.io/projected/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-kube-api-access-xlskv\") pod \"maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86\" (UID: \"b5f84bc4-6b63-436f-ad7c-492b0d2726b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" Jul 09 19:15:41.275015 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.275020 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86\" (UID: \"b5f84bc4-6b63-436f-ad7c-492b0d2726b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" Jul 09 19:15:41.275262 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.275154 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-maas-api-tls\") pod \"maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86\" (UID: \"b5f84bc4-6b63-436f-ad7c-492b0d2726b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" Jul 09 19:15:41.310291 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.310264 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" Jul 09 19:15:41.375998 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.375965 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-maas-api-tls\") pod \"maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86\" (UID: \"b5f84bc4-6b63-436f-ad7c-492b0d2726b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" Jul 09 19:15:41.376199 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.376026 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlskv\" (UniqueName: \"kubernetes.io/projected/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-kube-api-access-xlskv\") pod \"maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86\" (UID: \"b5f84bc4-6b63-436f-ad7c-492b0d2726b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" Jul 09 19:15:41.376199 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.376047 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86\" (UID: \"b5f84bc4-6b63-436f-ad7c-492b0d2726b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" Jul 09 19:15:41.376801 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.376773 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86\" (UID: \"b5f84bc4-6b63-436f-ad7c-492b0d2726b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" Jul 09 19:15:41.378756 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.378732 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-maas-api-tls\") pod \"maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86\" (UID: \"b5f84bc4-6b63-436f-ad7c-492b0d2726b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" Jul 09 19:15:41.386281 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.386257 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlskv\" (UniqueName: \"kubernetes.io/projected/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-kube-api-access-xlskv\") pod \"maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86\" (UID: \"b5f84bc4-6b63-436f-ad7c-492b0d2726b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" Jul 09 19:15:41.462001 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.461966 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" Jul 09 19:15:41.602189 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:41.602126 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86"] Jul 09 19:15:41.604424 ip-10-0-143-175 kubenswrapper[2585]: W0709 19:15:41.604388 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f84bc4_6b63_436f_ad7c_492b0d2726b9.slice/crio-1c00de1fd76fb98a56a25ed27dd773d0a462b76457d06378e4aa00634578d46d WatchSource:0}: Error finding container 1c00de1fd76fb98a56a25ed27dd773d0a462b76457d06378e4aa00634578d46d: Status 404 returned error can't find the container with id 1c00de1fd76fb98a56a25ed27dd773d0a462b76457d06378e4aa00634578d46d Jul 09 19:15:42.333306 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:42.333215 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" event={"ID":"b5f84bc4-6b63-436f-ad7c-492b0d2726b9","Type":"ContainerStarted","Data":"e44945a550bb327995f140302a7aeb543a08c8d62279b48a34760b94b52f6897"} Jul 09 19:15:42.333306 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:42.333265 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" event={"ID":"b5f84bc4-6b63-436f-ad7c-492b0d2726b9","Type":"ContainerStarted","Data":"1c00de1fd76fb98a56a25ed27dd773d0a462b76457d06378e4aa00634578d46d"} Jul 09 19:15:42.333306 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:42.333292 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" Jul 09 19:15:42.357955 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:42.357902 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" podStartSLOduration=0.973251298 podStartE2EDuration="1.357884069s" totalImagesPullingTime="384.632771ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:15:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:15:42.355492971 +0000 UTC m=+2059.099476085" watchObservedRunningTime="2026-07-09 19:15:42.357884069 +0000 UTC m=+2059.101867182" Jul 09 19:15:48.344332 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:48.344247 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" Jul 09 19:15:59.074484 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.074448 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx"] Jul 09 19:15:59.078151 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.078133 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.081878 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.081857 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-575766\"/\"kube-root-ca.crt\"" Jul 09 19:15:59.082131 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.082116 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-575766\"/\"default-dockercfg-ghg2p\"" Jul 09 19:15:59.082910 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.082892 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-575766\"/\"test-model-575766-kserve-self-signed-certs\"" Jul 09 19:15:59.083013 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.082883 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-575766\"/\"openshift-service-ca.crt\"" Jul 09 19:15:59.094924 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.094896 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx"] Jul 09 19:15:59.242368 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.242333 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb3232eb-5764-4c06-ac59-6158f8a547b8-tls-certs\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.242549 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.242376 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5lzl\" (UniqueName: \"kubernetes.io/projected/bb3232eb-5764-4c06-ac59-6158f8a547b8-kube-api-access-n5lzl\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.242549 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.242404 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-dshm\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.242549 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.242514 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-tmp-dir\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.242710 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.242552 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-home\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.242710 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.242572 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-model-cache\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.343540 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.343450 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-dshm\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.343540 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.343511 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-tmp-dir\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.343540 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.343533 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-home\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.343828 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.343550 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-model-cache\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.343828 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.343643 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb3232eb-5764-4c06-ac59-6158f8a547b8-tls-certs\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.343828 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.343666 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5lzl\" (UniqueName: \"kubernetes.io/projected/bb3232eb-5764-4c06-ac59-6158f8a547b8-kube-api-access-n5lzl\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.344003 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.343981 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-home\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.344069 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.344034 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-tmp-dir\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.344127 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.344062 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-model-cache\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.346020 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.345996 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-dshm\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.346411 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.346389 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb3232eb-5764-4c06-ac59-6158f8a547b8-tls-certs\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.355506 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.355483 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5lzl\" (UniqueName: \"kubernetes.io/projected/bb3232eb-5764-4c06-ac59-6158f8a547b8-kube-api-access-n5lzl\") pod \"test-model-575766-kserve-cb76f8969-v8mxx\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.395302 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.395264 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:15:59.548400 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:15:59.548324 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx"] Jul 09 19:15:59.551183 ip-10-0-143-175 kubenswrapper[2585]: W0709 19:15:59.551154 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb3232eb_5764_4c06_ac59_6158f8a547b8.slice/crio-372abb0a55f10713e7969f91c45d82ec1fc2a9d55aa61c47959f2356c5792117 WatchSource:0}: Error finding container 372abb0a55f10713e7969f91c45d82ec1fc2a9d55aa61c47959f2356c5792117: Status 404 returned error can't find the container with id 372abb0a55f10713e7969f91c45d82ec1fc2a9d55aa61c47959f2356c5792117 Jul 09 19:16:00.423822 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:00.423787 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" event={"ID":"bb3232eb-5764-4c06-ac59-6158f8a547b8","Type":"ContainerStarted","Data":"032e671ddff5b8f6fc23919613b6bdd65c2cbbd1811295084e838ff49fc90503"} Jul 09 19:16:00.424206 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:00.423830 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" event={"ID":"bb3232eb-5764-4c06-ac59-6158f8a547b8","Type":"ContainerStarted","Data":"372abb0a55f10713e7969f91c45d82ec1fc2a9d55aa61c47959f2356c5792117"} Jul 09 19:16:00.450317 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:00.450254 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" podStartSLOduration=1.450232845 podStartE2EDuration="1.450232845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:15:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:16:00.448969786 +0000 UTC m=+2077.192952900" watchObservedRunningTime="2026-07-09 19:16:00.450232845 +0000 UTC m=+2077.194215960" Jul 09 19:16:09.396035 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:09.396004 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:16:09.396557 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:09.396198 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:16:09.408297 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:09.408269 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:16:09.476391 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:09.476363 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:16:24.546406 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:24.546295 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 19:16:24.550845 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:24.549178 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 19:16:47.515019 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.514983 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz"] Jul 09 19:16:47.519289 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.519264 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" Jul 09 19:16:47.523072 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.523045 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-reuse-8e4d4c-serving-cert\"" Jul 09 19:16:47.532029 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.532005 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz"] Jul 09 19:16:47.595977 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.595942 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d550602-6360-499c-8f78-2be0f57b95d6-maas-api-tls\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-smthz\" (UID: \"6d550602-6360-499c-8f78-2be0f57b95d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" Jul 09 19:16:47.596171 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.596004 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g46b\" (UniqueName: \"kubernetes.io/projected/6d550602-6360-499c-8f78-2be0f57b95d6-kube-api-access-5g46b\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-smthz\" (UID: \"6d550602-6360-499c-8f78-2be0f57b95d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" Jul 09 19:16:47.596171 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.596029 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d550602-6360-499c-8f78-2be0f57b95d6-openshift-service-ca\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-smthz\" (UID: \"6d550602-6360-499c-8f78-2be0f57b95d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" Jul 09 19:16:47.696677 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.696639 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d550602-6360-499c-8f78-2be0f57b95d6-maas-api-tls\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-smthz\" (UID: \"6d550602-6360-499c-8f78-2be0f57b95d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" Jul 09 19:16:47.696875 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.696727 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g46b\" (UniqueName: \"kubernetes.io/projected/6d550602-6360-499c-8f78-2be0f57b95d6-kube-api-access-5g46b\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-smthz\" (UID: \"6d550602-6360-499c-8f78-2be0f57b95d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" Jul 09 19:16:47.696875 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.696760 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d550602-6360-499c-8f78-2be0f57b95d6-openshift-service-ca\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-smthz\" (UID: \"6d550602-6360-499c-8f78-2be0f57b95d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" Jul 09 19:16:47.697414 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.697389 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d550602-6360-499c-8f78-2be0f57b95d6-openshift-service-ca\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-smthz\" (UID: \"6d550602-6360-499c-8f78-2be0f57b95d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" Jul 09 19:16:47.699326 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.699299 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d550602-6360-499c-8f78-2be0f57b95d6-maas-api-tls\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-smthz\" (UID: \"6d550602-6360-499c-8f78-2be0f57b95d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" Jul 09 19:16:47.706722 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.706697 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g46b\" (UniqueName: \"kubernetes.io/projected/6d550602-6360-499c-8f78-2be0f57b95d6-kube-api-access-5g46b\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-smthz\" (UID: \"6d550602-6360-499c-8f78-2be0f57b95d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" Jul 09 19:16:47.832882 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.832799 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" Jul 09 19:16:47.977388 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.977365 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz"] Jul 09 19:16:47.979110 ip-10-0-143-175 kubenswrapper[2585]: W0709 19:16:47.979084 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d550602_6360_499c_8f78_2be0f57b95d6.slice/crio-951d4086b7cdef58256183b2e8c2744860a180353124678f6c0124b2513f48d8 WatchSource:0}: Error finding container 951d4086b7cdef58256183b2e8c2744860a180353124678f6c0124b2513f48d8: Status 404 returned error can't find the container with id 951d4086b7cdef58256183b2e8c2744860a180353124678f6c0124b2513f48d8 Jul 09 19:16:47.980381 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:47.980360 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 19:16:48.654854 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:48.654771 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" event={"ID":"6d550602-6360-499c-8f78-2be0f57b95d6","Type":"ContainerStarted","Data":"d08f1ea8c3b4e999a4bf7a3d03455b52ab9dca8d698a4b68df63cee5dc7c4b11"} Jul 09 19:16:48.654854 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:48.654826 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" event={"ID":"6d550602-6360-499c-8f78-2be0f57b95d6","Type":"ContainerStarted","Data":"951d4086b7cdef58256183b2e8c2744860a180353124678f6c0124b2513f48d8"} Jul 09 19:16:48.655229 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:48.654942 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" Jul 09 19:16:48.676987 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:48.676944 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" podStartSLOduration=1.304297977 podStartE2EDuration="1.676929842s" totalImagesPullingTime="372.631865ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:16:48.675700208 +0000 UTC m=+2125.419683323" watchObservedRunningTime="2026-07-09 19:16:48.676929842 +0000 UTC m=+2125.420912956" Jul 09 19:16:54.665167 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:16:54.665132 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" Jul 09 19:17:28.731562 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:28.731457 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz"] Jul 09 19:17:28.732169 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:28.731781 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" podUID="6d550602-6360-499c-8f78-2be0f57b95d6" containerName="maas-api" containerID="cri-o://d08f1ea8c3b4e999a4bf7a3d03455b52ab9dca8d698a4b68df63cee5dc7c4b11" gracePeriod=30 Jul 09 19:17:28.845794 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:28.845766 2585 generic.go:356] "Generic (PLEG): container finished" podID="6d550602-6360-499c-8f78-2be0f57b95d6" containerID="d08f1ea8c3b4e999a4bf7a3d03455b52ab9dca8d698a4b68df63cee5dc7c4b11" exitCode=0 Jul 09 19:17:28.845909 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:28.845839 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" event={"ID":"6d550602-6360-499c-8f78-2be0f57b95d6","Type":"ContainerDied","Data":"d08f1ea8c3b4e999a4bf7a3d03455b52ab9dca8d698a4b68df63cee5dc7c4b11"} Jul 09 19:17:28.976465 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:28.976442 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" Jul 09 19:17:29.082273 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:29.082198 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d550602-6360-499c-8f78-2be0f57b95d6-maas-api-tls\") pod \"6d550602-6360-499c-8f78-2be0f57b95d6\" (UID: \"6d550602-6360-499c-8f78-2be0f57b95d6\") " Jul 09 19:17:29.082414 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:29.082285 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d550602-6360-499c-8f78-2be0f57b95d6-openshift-service-ca\") pod \"6d550602-6360-499c-8f78-2be0f57b95d6\" (UID: \"6d550602-6360-499c-8f78-2be0f57b95d6\") " Jul 09 19:17:29.082414 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:29.082327 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5g46b\" (UniqueName: \"kubernetes.io/projected/6d550602-6360-499c-8f78-2be0f57b95d6-kube-api-access-5g46b\") pod \"6d550602-6360-499c-8f78-2be0f57b95d6\" (UID: \"6d550602-6360-499c-8f78-2be0f57b95d6\") " Jul 09 19:17:29.082665 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:29.082641 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d550602-6360-499c-8f78-2be0f57b95d6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6d550602-6360-499c-8f78-2be0f57b95d6" (UID: "6d550602-6360-499c-8f78-2be0f57b95d6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:17:29.084408 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:29.084383 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d550602-6360-499c-8f78-2be0f57b95d6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6d550602-6360-499c-8f78-2be0f57b95d6" (UID: "6d550602-6360-499c-8f78-2be0f57b95d6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:17:29.084520 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:29.084495 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d550602-6360-499c-8f78-2be0f57b95d6-kube-api-access-5g46b" (OuterVolumeSpecName: "kube-api-access-5g46b") pod "6d550602-6360-499c-8f78-2be0f57b95d6" (UID: "6d550602-6360-499c-8f78-2be0f57b95d6"). InnerVolumeSpecName "kube-api-access-5g46b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:17:29.183530 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:29.183493 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d550602-6360-499c-8f78-2be0f57b95d6-openshift-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:17:29.183530 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:29.183521 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g46b\" (UniqueName: \"kubernetes.io/projected/6d550602-6360-499c-8f78-2be0f57b95d6-kube-api-access-5g46b\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:17:29.183530 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:29.183532 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d550602-6360-499c-8f78-2be0f57b95d6-maas-api-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:17:29.852275 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:29.852248 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" event={"ID":"6d550602-6360-499c-8f78-2be0f57b95d6","Type":"ContainerDied","Data":"951d4086b7cdef58256183b2e8c2744860a180353124678f6c0124b2513f48d8"} Jul 09 19:17:29.852275 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:29.852272 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz" Jul 09 19:17:29.852727 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:29.852315 2585 scope.go:117] "RemoveContainer" containerID="d08f1ea8c3b4e999a4bf7a3d03455b52ab9dca8d698a4b68df63cee5dc7c4b11" Jul 09 19:17:29.871193 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:29.871166 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz"] Jul 09 19:17:29.876096 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:29.876072 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-smthz"] Jul 09 19:17:31.842548 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:17:31.842513 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d550602-6360-499c-8f78-2be0f57b95d6" path="/var/lib/kubelet/pods/6d550602-6360-499c-8f78-2be0f57b95d6/volumes" Jul 09 19:18:21.322006 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.321974 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq"] Jul 09 19:18:21.322573 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.322382 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d550602-6360-499c-8f78-2be0f57b95d6" containerName="maas-api" Jul 09 19:18:21.322573 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.322395 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d550602-6360-499c-8f78-2be0f57b95d6" containerName="maas-api" Jul 09 19:18:21.322573 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.322471 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d550602-6360-499c-8f78-2be0f57b95d6" containerName="maas-api" Jul 09 19:18:21.325817 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.325797 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" Jul 09 19:18:21.328502 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.328472 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-reuse-8e4d4c-serving-cert\"" Jul 09 19:18:21.341137 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.341116 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq"] Jul 09 19:18:21.366201 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.366180 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b184cc5-3452-4da1-829e-0a75ac852e98-maas-api-tls\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq\" (UID: \"6b184cc5-3452-4da1-829e-0a75ac852e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" Jul 09 19:18:21.366336 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.366216 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn96n\" (UniqueName: \"kubernetes.io/projected/6b184cc5-3452-4da1-829e-0a75ac852e98-kube-api-access-jn96n\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq\" (UID: \"6b184cc5-3452-4da1-829e-0a75ac852e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" Jul 09 19:18:21.366336 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.366297 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b184cc5-3452-4da1-829e-0a75ac852e98-openshift-service-ca\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq\" (UID: \"6b184cc5-3452-4da1-829e-0a75ac852e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" Jul 09 19:18:21.470263 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.470220 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b184cc5-3452-4da1-829e-0a75ac852e98-openshift-service-ca\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq\" (UID: \"6b184cc5-3452-4da1-829e-0a75ac852e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" Jul 09 19:18:21.470450 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.470335 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b184cc5-3452-4da1-829e-0a75ac852e98-maas-api-tls\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq\" (UID: \"6b184cc5-3452-4da1-829e-0a75ac852e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" Jul 09 19:18:21.470450 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.470379 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn96n\" (UniqueName: \"kubernetes.io/projected/6b184cc5-3452-4da1-829e-0a75ac852e98-kube-api-access-jn96n\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq\" (UID: \"6b184cc5-3452-4da1-829e-0a75ac852e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" Jul 09 19:18:21.471105 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.471079 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b184cc5-3452-4da1-829e-0a75ac852e98-openshift-service-ca\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq\" (UID: \"6b184cc5-3452-4da1-829e-0a75ac852e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" Jul 09 19:18:21.474213 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.474142 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b184cc5-3452-4da1-829e-0a75ac852e98-maas-api-tls\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq\" (UID: \"6b184cc5-3452-4da1-829e-0a75ac852e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" Jul 09 19:18:21.483945 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.483917 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn96n\" (UniqueName: \"kubernetes.io/projected/6b184cc5-3452-4da1-829e-0a75ac852e98-kube-api-access-jn96n\") pod \"maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq\" (UID: \"6b184cc5-3452-4da1-829e-0a75ac852e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" Jul 09 19:18:21.636373 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.636284 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" Jul 09 19:18:21.777698 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:21.777667 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq"] Jul 09 19:18:21.780963 ip-10-0-143-175 kubenswrapper[2585]: W0709 19:18:21.780918 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b184cc5_3452_4da1_829e_0a75ac852e98.slice/crio-69e18e95b4951df62322fb154110fac53a0148b0c8318601dff5aa7436b3c1c1 WatchSource:0}: Error finding container 69e18e95b4951df62322fb154110fac53a0148b0c8318601dff5aa7436b3c1c1: Status 404 returned error can't find the container with id 69e18e95b4951df62322fb154110fac53a0148b0c8318601dff5aa7436b3c1c1 Jul 09 19:18:22.102558 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:22.102518 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" event={"ID":"6b184cc5-3452-4da1-829e-0a75ac852e98","Type":"ContainerStarted","Data":"69e18e95b4951df62322fb154110fac53a0148b0c8318601dff5aa7436b3c1c1"} Jul 09 19:18:23.110150 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:23.110107 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" event={"ID":"6b184cc5-3452-4da1-829e-0a75ac852e98","Type":"ContainerStarted","Data":"1307d892a0c43bca88ac3f44f5ef686ed54f34785a794753994e45e97b2591f1"} Jul 09 19:18:23.110533 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:23.110239 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" Jul 09 19:18:23.132124 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:23.132069 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" podStartSLOduration=1.748047368 podStartE2EDuration="2.132052078s" totalImagesPullingTime="384.00471ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:18:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:18:23.130733413 +0000 UTC m=+2219.874716551" watchObservedRunningTime="2026-07-09 19:18:23.132052078 +0000 UTC m=+2219.876035191" Jul 09 19:18:29.120531 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:18:29.120501 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" Jul 09 19:19:02.514773 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:02.514676 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq"] Jul 09 19:19:02.515321 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:02.514985 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" podUID="6b184cc5-3452-4da1-829e-0a75ac852e98" containerName="maas-api" containerID="cri-o://1307d892a0c43bca88ac3f44f5ef686ed54f34785a794753994e45e97b2591f1" gracePeriod=30 Jul 09 19:19:02.759814 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:02.759789 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" Jul 09 19:19:02.850832 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:02.850750 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b184cc5-3452-4da1-829e-0a75ac852e98-maas-api-tls\") pod \"6b184cc5-3452-4da1-829e-0a75ac852e98\" (UID: \"6b184cc5-3452-4da1-829e-0a75ac852e98\") " Jul 09 19:19:02.850832 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:02.850822 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn96n\" (UniqueName: \"kubernetes.io/projected/6b184cc5-3452-4da1-829e-0a75ac852e98-kube-api-access-jn96n\") pod \"6b184cc5-3452-4da1-829e-0a75ac852e98\" (UID: \"6b184cc5-3452-4da1-829e-0a75ac852e98\") " Jul 09 19:19:02.851023 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:02.850911 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b184cc5-3452-4da1-829e-0a75ac852e98-openshift-service-ca\") pod \"6b184cc5-3452-4da1-829e-0a75ac852e98\" (UID: \"6b184cc5-3452-4da1-829e-0a75ac852e98\") " Jul 09 19:19:02.851299 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:02.851273 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b184cc5-3452-4da1-829e-0a75ac852e98-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6b184cc5-3452-4da1-829e-0a75ac852e98" (UID: "6b184cc5-3452-4da1-829e-0a75ac852e98"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:19:02.853011 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:02.852983 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b184cc5-3452-4da1-829e-0a75ac852e98-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6b184cc5-3452-4da1-829e-0a75ac852e98" (UID: "6b184cc5-3452-4da1-829e-0a75ac852e98"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:19:02.853100 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:02.853035 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b184cc5-3452-4da1-829e-0a75ac852e98-kube-api-access-jn96n" (OuterVolumeSpecName: "kube-api-access-jn96n") pod "6b184cc5-3452-4da1-829e-0a75ac852e98" (UID: "6b184cc5-3452-4da1-829e-0a75ac852e98"). InnerVolumeSpecName "kube-api-access-jn96n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:19:02.952639 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:02.952574 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b184cc5-3452-4da1-829e-0a75ac852e98-maas-api-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:19:02.952639 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:02.952633 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn96n\" (UniqueName: \"kubernetes.io/projected/6b184cc5-3452-4da1-829e-0a75ac852e98-kube-api-access-jn96n\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:19:02.952639 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:02.952647 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b184cc5-3452-4da1-829e-0a75ac852e98-openshift-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:19:03.297825 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:03.297791 2585 generic.go:356] "Generic (PLEG): container finished" podID="6b184cc5-3452-4da1-829e-0a75ac852e98" containerID="1307d892a0c43bca88ac3f44f5ef686ed54f34785a794753994e45e97b2591f1" exitCode=0 Jul 09 19:19:03.298000 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:03.297839 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" Jul 09 19:19:03.298000 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:03.297887 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" event={"ID":"6b184cc5-3452-4da1-829e-0a75ac852e98","Type":"ContainerDied","Data":"1307d892a0c43bca88ac3f44f5ef686ed54f34785a794753994e45e97b2591f1"} Jul 09 19:19:03.298000 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:03.297942 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq" event={"ID":"6b184cc5-3452-4da1-829e-0a75ac852e98","Type":"ContainerDied","Data":"69e18e95b4951df62322fb154110fac53a0148b0c8318601dff5aa7436b3c1c1"} Jul 09 19:19:03.298000 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:03.297986 2585 scope.go:117] "RemoveContainer" containerID="1307d892a0c43bca88ac3f44f5ef686ed54f34785a794753994e45e97b2591f1" Jul 09 19:19:03.307405 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:03.307386 2585 scope.go:117] "RemoveContainer" containerID="1307d892a0c43bca88ac3f44f5ef686ed54f34785a794753994e45e97b2591f1" Jul 09 19:19:03.307718 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:19:03.307697 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1307d892a0c43bca88ac3f44f5ef686ed54f34785a794753994e45e97b2591f1\": container with ID starting with 1307d892a0c43bca88ac3f44f5ef686ed54f34785a794753994e45e97b2591f1 not found: ID does not exist" containerID="1307d892a0c43bca88ac3f44f5ef686ed54f34785a794753994e45e97b2591f1" Jul 09 19:19:03.307782 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:03.307728 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1307d892a0c43bca88ac3f44f5ef686ed54f34785a794753994e45e97b2591f1"} err="failed to get container status \"1307d892a0c43bca88ac3f44f5ef686ed54f34785a794753994e45e97b2591f1\": rpc error: code = NotFound desc = could not find container \"1307d892a0c43bca88ac3f44f5ef686ed54f34785a794753994e45e97b2591f1\": container with ID starting with 1307d892a0c43bca88ac3f44f5ef686ed54f34785a794753994e45e97b2591f1 not found: ID does not exist" Jul 09 19:19:03.326281 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:03.326257 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq"] Jul 09 19:19:03.327845 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:03.327826 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-8e4d4c-85697d7974-5btnq"] Jul 09 19:19:03.844210 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:19:03.844176 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b184cc5-3452-4da1-829e-0a75ac852e98" path="/var/lib/kubelet/pods/6b184cc5-3452-4da1-829e-0a75ac852e98/volumes" Jul 09 19:20:08.874746 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:08.874715 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx"] Jul 09 19:20:10.828211 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:10.828173 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs"] Jul 09 19:20:10.828730 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:10.828556 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" podUID="d22cafa1-ce03-411e-a3c4-314fcb70178f" containerName="maas-api" containerID="cri-o://df909ff3d1303b144efba584faef7368db02974cce7502b5c7b1b2abc916d98a" gracePeriod=30 Jul 09 19:20:11.074103 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.074080 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" Jul 09 19:20:11.167833 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.167750 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d22cafa1-ce03-411e-a3c4-314fcb70178f-maas-api-tls\") pod \"d22cafa1-ce03-411e-a3c4-314fcb70178f\" (UID: \"d22cafa1-ce03-411e-a3c4-314fcb70178f\") " Jul 09 19:20:11.167833 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.167798 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fgr4\" (UniqueName: \"kubernetes.io/projected/d22cafa1-ce03-411e-a3c4-314fcb70178f-kube-api-access-2fgr4\") pod \"d22cafa1-ce03-411e-a3c4-314fcb70178f\" (UID: \"d22cafa1-ce03-411e-a3c4-314fcb70178f\") " Jul 09 19:20:11.168071 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.167902 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d22cafa1-ce03-411e-a3c4-314fcb70178f-openshift-service-ca\") pod \"d22cafa1-ce03-411e-a3c4-314fcb70178f\" (UID: \"d22cafa1-ce03-411e-a3c4-314fcb70178f\") " Jul 09 19:20:11.168238 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.168211 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d22cafa1-ce03-411e-a3c4-314fcb70178f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d22cafa1-ce03-411e-a3c4-314fcb70178f" (UID: "d22cafa1-ce03-411e-a3c4-314fcb70178f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:20:11.170038 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.170013 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d22cafa1-ce03-411e-a3c4-314fcb70178f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d22cafa1-ce03-411e-a3c4-314fcb70178f" (UID: "d22cafa1-ce03-411e-a3c4-314fcb70178f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:20:11.170038 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.170027 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d22cafa1-ce03-411e-a3c4-314fcb70178f-kube-api-access-2fgr4" (OuterVolumeSpecName: "kube-api-access-2fgr4") pod "d22cafa1-ce03-411e-a3c4-314fcb70178f" (UID: "d22cafa1-ce03-411e-a3c4-314fcb70178f"). InnerVolumeSpecName "kube-api-access-2fgr4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:20:11.269206 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.269169 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d22cafa1-ce03-411e-a3c4-314fcb70178f-openshift-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:20:11.269206 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.269200 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d22cafa1-ce03-411e-a3c4-314fcb70178f-maas-api-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:20:11.269206 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.269210 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fgr4\" (UniqueName: \"kubernetes.io/projected/d22cafa1-ce03-411e-a3c4-314fcb70178f-kube-api-access-2fgr4\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:20:11.628863 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.628826 2585 generic.go:356] "Generic (PLEG): container finished" podID="d22cafa1-ce03-411e-a3c4-314fcb70178f" containerID="df909ff3d1303b144efba584faef7368db02974cce7502b5c7b1b2abc916d98a" exitCode=0 Jul 09 19:20:11.628863 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.628863 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" Jul 09 19:20:11.629188 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.628906 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" event={"ID":"d22cafa1-ce03-411e-a3c4-314fcb70178f","Type":"ContainerDied","Data":"df909ff3d1303b144efba584faef7368db02974cce7502b5c7b1b2abc916d98a"} Jul 09 19:20:11.629188 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.628954 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs" event={"ID":"d22cafa1-ce03-411e-a3c4-314fcb70178f","Type":"ContainerDied","Data":"e5ddf483decc86137477ae27b17502797c6ba866d316903244058d4596bf723a"} Jul 09 19:20:11.629188 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.628978 2585 scope.go:117] "RemoveContainer" containerID="df909ff3d1303b144efba584faef7368db02974cce7502b5c7b1b2abc916d98a" Jul 09 19:20:11.638390 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.638366 2585 scope.go:117] "RemoveContainer" containerID="df909ff3d1303b144efba584faef7368db02974cce7502b5c7b1b2abc916d98a" Jul 09 19:20:11.638696 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:20:11.638677 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df909ff3d1303b144efba584faef7368db02974cce7502b5c7b1b2abc916d98a\": container with ID starting with df909ff3d1303b144efba584faef7368db02974cce7502b5c7b1b2abc916d98a not found: ID does not exist" containerID="df909ff3d1303b144efba584faef7368db02974cce7502b5c7b1b2abc916d98a" Jul 09 19:20:11.638756 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.638704 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df909ff3d1303b144efba584faef7368db02974cce7502b5c7b1b2abc916d98a"} err="failed to get container status \"df909ff3d1303b144efba584faef7368db02974cce7502b5c7b1b2abc916d98a\": rpc error: code = NotFound desc = could not find container \"df909ff3d1303b144efba584faef7368db02974cce7502b5c7b1b2abc916d98a\": container with ID starting with df909ff3d1303b144efba584faef7368db02974cce7502b5c7b1b2abc916d98a not found: ID does not exist" Jul 09 19:20:11.651652 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.651624 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs"] Jul 09 19:20:11.656292 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.656267 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-575766-6556c67765-zqgjs"] Jul 09 19:20:11.843184 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:11.843153 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d22cafa1-ce03-411e-a3c4-314fcb70178f" path="/var/lib/kubelet/pods/d22cafa1-ce03-411e-a3c4-314fcb70178f/volumes" Jul 09 19:20:23.895736 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:23.895631 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" podUID="bb3232eb-5764-4c06-ac59-6158f8a547b8" containerName="main" containerID="cri-o://032e671ddff5b8f6fc23919613b6bdd65c2cbbd1811295084e838ff49fc90503" gracePeriod=45 Jul 09 19:20:24.150143 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.150076 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:20:24.190497 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.190456 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-home\") pod \"bb3232eb-5764-4c06-ac59-6158f8a547b8\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " Jul 09 19:20:24.190686 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.190517 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb3232eb-5764-4c06-ac59-6158f8a547b8-tls-certs\") pod \"bb3232eb-5764-4c06-ac59-6158f8a547b8\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " Jul 09 19:20:24.190686 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.190598 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-model-cache\") pod \"bb3232eb-5764-4c06-ac59-6158f8a547b8\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " Jul 09 19:20:24.190686 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.190629 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-tmp-dir\") pod \"bb3232eb-5764-4c06-ac59-6158f8a547b8\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " Jul 09 19:20:24.190686 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.190681 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5lzl\" (UniqueName: \"kubernetes.io/projected/bb3232eb-5764-4c06-ac59-6158f8a547b8-kube-api-access-n5lzl\") pod \"bb3232eb-5764-4c06-ac59-6158f8a547b8\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " Jul 09 19:20:24.190914 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.190715 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-dshm\") pod \"bb3232eb-5764-4c06-ac59-6158f8a547b8\" (UID: \"bb3232eb-5764-4c06-ac59-6158f8a547b8\") " Jul 09 19:20:24.190914 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.190754 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-home" (OuterVolumeSpecName: "home") pod "bb3232eb-5764-4c06-ac59-6158f8a547b8" (UID: "bb3232eb-5764-4c06-ac59-6158f8a547b8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:20:24.190914 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.190869 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-model-cache" (OuterVolumeSpecName: "model-cache") pod "bb3232eb-5764-4c06-ac59-6158f8a547b8" (UID: "bb3232eb-5764-4c06-ac59-6158f8a547b8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:20:24.191074 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.190924 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bb3232eb-5764-4c06-ac59-6158f8a547b8" (UID: "bb3232eb-5764-4c06-ac59-6158f8a547b8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:20:24.191074 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.191044 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-home\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:20:24.191074 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.191061 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-model-cache\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:20:24.191203 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.191076 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-tmp-dir\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:20:24.193350 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.193285 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb3232eb-5764-4c06-ac59-6158f8a547b8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bb3232eb-5764-4c06-ac59-6158f8a547b8" (UID: "bb3232eb-5764-4c06-ac59-6158f8a547b8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:20:24.193350 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.193328 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb3232eb-5764-4c06-ac59-6158f8a547b8-kube-api-access-n5lzl" (OuterVolumeSpecName: "kube-api-access-n5lzl") pod "bb3232eb-5764-4c06-ac59-6158f8a547b8" (UID: "bb3232eb-5764-4c06-ac59-6158f8a547b8"). InnerVolumeSpecName "kube-api-access-n5lzl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:20:24.193523 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.193392 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-dshm" (OuterVolumeSpecName: "dshm") pod "bb3232eb-5764-4c06-ac59-6158f8a547b8" (UID: "bb3232eb-5764-4c06-ac59-6158f8a547b8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:20:24.291692 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.291654 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5lzl\" (UniqueName: \"kubernetes.io/projected/bb3232eb-5764-4c06-ac59-6158f8a547b8-kube-api-access-n5lzl\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:20:24.291692 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.291687 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb3232eb-5764-4c06-ac59-6158f8a547b8-dshm\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:20:24.291692 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.291699 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb3232eb-5764-4c06-ac59-6158f8a547b8-tls-certs\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:20:24.692801 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.692767 2585 generic.go:356] "Generic (PLEG): container finished" podID="bb3232eb-5764-4c06-ac59-6158f8a547b8" containerID="032e671ddff5b8f6fc23919613b6bdd65c2cbbd1811295084e838ff49fc90503" exitCode=0 Jul 09 19:20:24.692947 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.692810 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" Jul 09 19:20:24.692947 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.692815 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" event={"ID":"bb3232eb-5764-4c06-ac59-6158f8a547b8","Type":"ContainerDied","Data":"032e671ddff5b8f6fc23919613b6bdd65c2cbbd1811295084e838ff49fc90503"} Jul 09 19:20:24.692947 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.692846 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx" event={"ID":"bb3232eb-5764-4c06-ac59-6158f8a547b8","Type":"ContainerDied","Data":"372abb0a55f10713e7969f91c45d82ec1fc2a9d55aa61c47959f2356c5792117"} Jul 09 19:20:24.692947 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.692862 2585 scope.go:117] "RemoveContainer" containerID="032e671ddff5b8f6fc23919613b6bdd65c2cbbd1811295084e838ff49fc90503" Jul 09 19:20:24.702281 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.702259 2585 scope.go:117] "RemoveContainer" containerID="032e671ddff5b8f6fc23919613b6bdd65c2cbbd1811295084e838ff49fc90503" Jul 09 19:20:24.702527 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:20:24.702509 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"032e671ddff5b8f6fc23919613b6bdd65c2cbbd1811295084e838ff49fc90503\": container with ID starting with 032e671ddff5b8f6fc23919613b6bdd65c2cbbd1811295084e838ff49fc90503 not found: ID does not exist" containerID="032e671ddff5b8f6fc23919613b6bdd65c2cbbd1811295084e838ff49fc90503" Jul 09 19:20:24.702616 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.702534 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"032e671ddff5b8f6fc23919613b6bdd65c2cbbd1811295084e838ff49fc90503"} err="failed to get container status \"032e671ddff5b8f6fc23919613b6bdd65c2cbbd1811295084e838ff49fc90503\": rpc error: code = NotFound desc = could not find container \"032e671ddff5b8f6fc23919613b6bdd65c2cbbd1811295084e838ff49fc90503\": container with ID starting with 032e671ddff5b8f6fc23919613b6bdd65c2cbbd1811295084e838ff49fc90503 not found: ID does not exist" Jul 09 19:20:24.716701 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.716668 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx"] Jul 09 19:20:24.725509 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:24.725481 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-575766/test-model-575766-kserve-cb76f8969-v8mxx"] Jul 09 19:20:25.843240 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:25.843209 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb3232eb-5764-4c06-ac59-6158f8a547b8" path="/var/lib/kubelet/pods/bb3232eb-5764-4c06-ac59-6158f8a547b8/volumes" Jul 09 19:20:54.755215 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:54.755182 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86"] Jul 09 19:20:54.755821 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:54.755401 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" podUID="b5f84bc4-6b63-436f-ad7c-492b0d2726b9" containerName="maas-api" containerID="cri-o://e44945a550bb327995f140302a7aeb543a08c8d62279b48a34760b94b52f6897" gracePeriod=30 Jul 09 19:20:55.003291 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.003264 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" Jul 09 19:20:55.165115 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.165036 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-openshift-service-ca\") pod \"b5f84bc4-6b63-436f-ad7c-492b0d2726b9\" (UID: \"b5f84bc4-6b63-436f-ad7c-492b0d2726b9\") " Jul 09 19:20:55.165115 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.165091 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-maas-api-tls\") pod \"b5f84bc4-6b63-436f-ad7c-492b0d2726b9\" (UID: \"b5f84bc4-6b63-436f-ad7c-492b0d2726b9\") " Jul 09 19:20:55.165305 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.165153 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlskv\" (UniqueName: \"kubernetes.io/projected/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-kube-api-access-xlskv\") pod \"b5f84bc4-6b63-436f-ad7c-492b0d2726b9\" (UID: \"b5f84bc4-6b63-436f-ad7c-492b0d2726b9\") " Jul 09 19:20:55.165417 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.165395 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b5f84bc4-6b63-436f-ad7c-492b0d2726b9" (UID: "b5f84bc4-6b63-436f-ad7c-492b0d2726b9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:20:55.167290 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.167260 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-kube-api-access-xlskv" (OuterVolumeSpecName: "kube-api-access-xlskv") pod "b5f84bc4-6b63-436f-ad7c-492b0d2726b9" (UID: "b5f84bc4-6b63-436f-ad7c-492b0d2726b9"). InnerVolumeSpecName "kube-api-access-xlskv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:20:55.167408 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.167337 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b5f84bc4-6b63-436f-ad7c-492b0d2726b9" (UID: "b5f84bc4-6b63-436f-ad7c-492b0d2726b9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:20:55.266301 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.266262 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlskv\" (UniqueName: \"kubernetes.io/projected/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-kube-api-access-xlskv\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:20:55.266301 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.266294 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-openshift-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:20:55.266501 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.266309 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5f84bc4-6b63-436f-ad7c-492b0d2726b9-maas-api-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:20:55.845758 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.845726 2585 generic.go:356] "Generic (PLEG): container finished" podID="b5f84bc4-6b63-436f-ad7c-492b0d2726b9" containerID="e44945a550bb327995f140302a7aeb543a08c8d62279b48a34760b94b52f6897" exitCode=0 Jul 09 19:20:55.846152 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.845852 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" Jul 09 19:20:55.848594 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.848558 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" event={"ID":"b5f84bc4-6b63-436f-ad7c-492b0d2726b9","Type":"ContainerDied","Data":"e44945a550bb327995f140302a7aeb543a08c8d62279b48a34760b94b52f6897"} Jul 09 19:20:55.848740 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.848629 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86" event={"ID":"b5f84bc4-6b63-436f-ad7c-492b0d2726b9","Type":"ContainerDied","Data":"1c00de1fd76fb98a56a25ed27dd773d0a462b76457d06378e4aa00634578d46d"} Jul 09 19:20:55.848740 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.848656 2585 scope.go:117] "RemoveContainer" containerID="e44945a550bb327995f140302a7aeb543a08c8d62279b48a34760b94b52f6897" Jul 09 19:20:55.858831 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.858814 2585 scope.go:117] "RemoveContainer" containerID="e44945a550bb327995f140302a7aeb543a08c8d62279b48a34760b94b52f6897" Jul 09 19:20:55.859077 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:20:55.859047 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e44945a550bb327995f140302a7aeb543a08c8d62279b48a34760b94b52f6897\": container with ID starting with e44945a550bb327995f140302a7aeb543a08c8d62279b48a34760b94b52f6897 not found: ID does not exist" containerID="e44945a550bb327995f140302a7aeb543a08c8d62279b48a34760b94b52f6897" Jul 09 19:20:55.859145 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.859084 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e44945a550bb327995f140302a7aeb543a08c8d62279b48a34760b94b52f6897"} err="failed to get container status \"e44945a550bb327995f140302a7aeb543a08c8d62279b48a34760b94b52f6897\": rpc error: code = NotFound desc = could not find container \"e44945a550bb327995f140302a7aeb543a08c8d62279b48a34760b94b52f6897\": container with ID starting with e44945a550bb327995f140302a7aeb543a08c8d62279b48a34760b94b52f6897 not found: ID does not exist" Jul 09 19:20:55.876254 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.876228 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86"] Jul 09 19:20:55.880575 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:55.880554 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8d3645-78ddb497d-vvv86"] Jul 09 19:20:57.842830 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:20:57.842799 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5f84bc4-6b63-436f-ad7c-492b0d2726b9" path="/var/lib/kubelet/pods/b5f84bc4-6b63-436f-ad7c-492b0d2726b9/volumes" Jul 09 19:21:04.696354 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.696309 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444"] Jul 09 19:21:04.696867 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.696851 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb3232eb-5764-4c06-ac59-6158f8a547b8" containerName="main" Jul 09 19:21:04.696926 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.696871 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb3232eb-5764-4c06-ac59-6158f8a547b8" containerName="main" Jul 09 19:21:04.696926 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.696886 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d22cafa1-ce03-411e-a3c4-314fcb70178f" containerName="maas-api" Jul 09 19:21:04.696926 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.696895 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d22cafa1-ce03-411e-a3c4-314fcb70178f" containerName="maas-api" Jul 09 19:21:04.696926 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.696912 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f84bc4-6b63-436f-ad7c-492b0d2726b9" containerName="maas-api" Jul 09 19:21:04.696926 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.696920 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f84bc4-6b63-436f-ad7c-492b0d2726b9" containerName="maas-api" Jul 09 19:21:04.697111 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.696952 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b184cc5-3452-4da1-829e-0a75ac852e98" containerName="maas-api" Jul 09 19:21:04.697111 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.696961 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b184cc5-3452-4da1-829e-0a75ac852e98" containerName="maas-api" Jul 09 19:21:04.697111 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.697036 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f84bc4-6b63-436f-ad7c-492b0d2726b9" containerName="maas-api" Jul 09 19:21:04.697111 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.697050 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb3232eb-5764-4c06-ac59-6158f8a547b8" containerName="main" Jul 09 19:21:04.697111 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.697059 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d22cafa1-ce03-411e-a3c4-314fcb70178f" containerName="maas-api" Jul 09 19:21:04.697111 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.697067 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b184cc5-3452-4da1-829e-0a75ac852e98" containerName="maas-api" Jul 09 19:21:04.700413 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.700382 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" Jul 09 19:21:04.705171 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.705150 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-2f44fe-serving-cert\"" Jul 09 19:21:04.712629 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.712600 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444"] Jul 09 19:21:04.849428 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.849385 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68847\" (UniqueName: \"kubernetes.io/projected/1da73102-a001-40ed-a38f-7c2ba7318644-kube-api-access-68847\") pod \"maas-api-e2e-api-a-2f44fe-799777c86f-jm444\" (UID: \"1da73102-a001-40ed-a38f-7c2ba7318644\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" Jul 09 19:21:04.849428 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.849431 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1da73102-a001-40ed-a38f-7c2ba7318644-maas-api-tls\") pod \"maas-api-e2e-api-a-2f44fe-799777c86f-jm444\" (UID: \"1da73102-a001-40ed-a38f-7c2ba7318644\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" Jul 09 19:21:04.849697 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.849508 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1da73102-a001-40ed-a38f-7c2ba7318644-openshift-service-ca\") pod \"maas-api-e2e-api-a-2f44fe-799777c86f-jm444\" (UID: \"1da73102-a001-40ed-a38f-7c2ba7318644\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" Jul 09 19:21:04.951005 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.950919 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1da73102-a001-40ed-a38f-7c2ba7318644-openshift-service-ca\") pod \"maas-api-e2e-api-a-2f44fe-799777c86f-jm444\" (UID: \"1da73102-a001-40ed-a38f-7c2ba7318644\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" Jul 09 19:21:04.951005 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.950998 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68847\" (UniqueName: \"kubernetes.io/projected/1da73102-a001-40ed-a38f-7c2ba7318644-kube-api-access-68847\") pod \"maas-api-e2e-api-a-2f44fe-799777c86f-jm444\" (UID: \"1da73102-a001-40ed-a38f-7c2ba7318644\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" Jul 09 19:21:04.951200 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.951036 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1da73102-a001-40ed-a38f-7c2ba7318644-maas-api-tls\") pod \"maas-api-e2e-api-a-2f44fe-799777c86f-jm444\" (UID: \"1da73102-a001-40ed-a38f-7c2ba7318644\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" Jul 09 19:21:04.951665 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.951645 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1da73102-a001-40ed-a38f-7c2ba7318644-openshift-service-ca\") pod \"maas-api-e2e-api-a-2f44fe-799777c86f-jm444\" (UID: \"1da73102-a001-40ed-a38f-7c2ba7318644\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" Jul 09 19:21:04.953919 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.953895 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1da73102-a001-40ed-a38f-7c2ba7318644-maas-api-tls\") pod \"maas-api-e2e-api-a-2f44fe-799777c86f-jm444\" (UID: \"1da73102-a001-40ed-a38f-7c2ba7318644\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" Jul 09 19:21:04.964680 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:04.964653 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68847\" (UniqueName: \"kubernetes.io/projected/1da73102-a001-40ed-a38f-7c2ba7318644-kube-api-access-68847\") pod \"maas-api-e2e-api-a-2f44fe-799777c86f-jm444\" (UID: \"1da73102-a001-40ed-a38f-7c2ba7318644\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" Jul 09 19:21:05.011558 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:05.011521 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" Jul 09 19:21:05.151768 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:05.151738 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444"] Jul 09 19:21:05.154069 ip-10-0-143-175 kubenswrapper[2585]: W0709 19:21:05.154034 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1da73102_a001_40ed_a38f_7c2ba7318644.slice/crio-aa48c62f4d69a9bf403886f731200879eec4321726c4c7f293f790e6462bd515 WatchSource:0}: Error finding container aa48c62f4d69a9bf403886f731200879eec4321726c4c7f293f790e6462bd515: Status 404 returned error can't find the container with id aa48c62f4d69a9bf403886f731200879eec4321726c4c7f293f790e6462bd515 Jul 09 19:21:05.897546 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:05.897437 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" event={"ID":"1da73102-a001-40ed-a38f-7c2ba7318644","Type":"ContainerStarted","Data":"d28f7b77df090a183752841a8dfd76a599087c34a6a12db69645e14f80328952"} Jul 09 19:21:05.897546 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:05.897502 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" event={"ID":"1da73102-a001-40ed-a38f-7c2ba7318644","Type":"ContainerStarted","Data":"aa48c62f4d69a9bf403886f731200879eec4321726c4c7f293f790e6462bd515"} Jul 09 19:21:05.897546 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:05.897531 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" Jul 09 19:21:05.933693 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:05.933645 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" podStartSLOduration=1.486192897 podStartE2EDuration="1.93362539s" totalImagesPullingTime="447.432493ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:21:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:21:05.932003007 +0000 UTC m=+2382.675986122" watchObservedRunningTime="2026-07-09 19:21:05.93362539 +0000 UTC m=+2382.677608506" Jul 09 19:21:11.907524 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:11.907496 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" Jul 09 19:21:13.022970 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:13.022932 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz"] Jul 09 19:21:13.036319 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:13.036283 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:13.039509 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:13.039483 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-0bfb2e-serving-cert\"" Jul 09 19:21:13.040212 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:13.040187 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz"] Jul 09 19:21:13.127947 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:13.127906 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bz28\" (UniqueName: \"kubernetes.io/projected/2269e98a-f421-43e2-b448-97458a1ecd49-kube-api-access-7bz28\") pod \"maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz\" (UID: \"2269e98a-f421-43e2-b448-97458a1ecd49\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:13.128147 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:13.127973 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2269e98a-f421-43e2-b448-97458a1ecd49-openshift-service-ca\") pod \"maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz\" (UID: \"2269e98a-f421-43e2-b448-97458a1ecd49\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:13.128147 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:13.128055 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2269e98a-f421-43e2-b448-97458a1ecd49-maas-api-tls\") pod \"maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz\" (UID: \"2269e98a-f421-43e2-b448-97458a1ecd49\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:13.228623 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:13.228565 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bz28\" (UniqueName: \"kubernetes.io/projected/2269e98a-f421-43e2-b448-97458a1ecd49-kube-api-access-7bz28\") pod \"maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz\" (UID: \"2269e98a-f421-43e2-b448-97458a1ecd49\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:13.228788 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:13.228688 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2269e98a-f421-43e2-b448-97458a1ecd49-openshift-service-ca\") pod \"maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz\" (UID: \"2269e98a-f421-43e2-b448-97458a1ecd49\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:13.228788 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:13.228750 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2269e98a-f421-43e2-b448-97458a1ecd49-maas-api-tls\") pod \"maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz\" (UID: \"2269e98a-f421-43e2-b448-97458a1ecd49\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:13.228905 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:21:13.228888 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-serving-cert: secret "maas-api-e2e-api-b-0bfb2e-serving-cert" not found Jul 09 19:21:13.228961 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:21:13.228951 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2269e98a-f421-43e2-b448-97458a1ecd49-maas-api-tls podName:2269e98a-f421-43e2-b448-97458a1ecd49 nodeName:}" failed. No retries permitted until 2026-07-09 19:21:13.728933999 +0000 UTC m=+2390.472917092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2269e98a-f421-43e2-b448-97458a1ecd49-maas-api-tls") pod "maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" (UID: "2269e98a-f421-43e2-b448-97458a1ecd49") : secret "maas-api-e2e-api-b-0bfb2e-serving-cert" not found Jul 09 19:21:13.229332 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:13.229314 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2269e98a-f421-43e2-b448-97458a1ecd49-openshift-service-ca\") pod \"maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz\" (UID: \"2269e98a-f421-43e2-b448-97458a1ecd49\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:13.238673 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:13.238646 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bz28\" (UniqueName: \"kubernetes.io/projected/2269e98a-f421-43e2-b448-97458a1ecd49-kube-api-access-7bz28\") pod \"maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz\" (UID: \"2269e98a-f421-43e2-b448-97458a1ecd49\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:13.734253 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:13.734217 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2269e98a-f421-43e2-b448-97458a1ecd49-maas-api-tls\") pod \"maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz\" (UID: \"2269e98a-f421-43e2-b448-97458a1ecd49\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:13.736843 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:13.736817 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2269e98a-f421-43e2-b448-97458a1ecd49-maas-api-tls\") pod \"maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz\" (UID: \"2269e98a-f421-43e2-b448-97458a1ecd49\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:13.953814 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:13.953764 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:14.102662 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:14.102637 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz"] Jul 09 19:21:14.105499 ip-10-0-143-175 kubenswrapper[2585]: W0709 19:21:14.105469 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2269e98a_f421_43e2_b448_97458a1ecd49.slice/crio-364bd50b06bac4efc70c52ae102d751ad3b21417838ff574311b957d3648fa2e WatchSource:0}: Error finding container 364bd50b06bac4efc70c52ae102d751ad3b21417838ff574311b957d3648fa2e: Status 404 returned error can't find the container with id 364bd50b06bac4efc70c52ae102d751ad3b21417838ff574311b957d3648fa2e Jul 09 19:21:14.942003 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:14.941972 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" event={"ID":"2269e98a-f421-43e2-b448-97458a1ecd49","Type":"ContainerStarted","Data":"1552290fe625d227586b5a939243dc9c451b1d95fbdbebf0a6cd77eb5fe77f50"} Jul 09 19:21:14.942212 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:14.942021 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" event={"ID":"2269e98a-f421-43e2-b448-97458a1ecd49","Type":"ContainerStarted","Data":"364bd50b06bac4efc70c52ae102d751ad3b21417838ff574311b957d3648fa2e"} Jul 09 19:21:14.942212 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:14.942115 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:14.963175 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:14.963124 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" podStartSLOduration=2.611854849 podStartE2EDuration="2.963107196s" totalImagesPullingTime="351.252347ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:21:14.961424146 +0000 UTC m=+2391.705407259" watchObservedRunningTime="2026-07-09 19:21:14.963107196 +0000 UTC m=+2391.707090339" Jul 09 19:21:20.953464 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:20.953431 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:24.581779 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:24.581657 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 19:21:24.599746 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:24.587914 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 19:21:26.419700 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:26.419667 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444"] Jul 09 19:21:26.420384 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:26.419920 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" podUID="1da73102-a001-40ed-a38f-7c2ba7318644" containerName="maas-api" containerID="cri-o://d28f7b77df090a183752841a8dfd76a599087c34a6a12db69645e14f80328952" gracePeriod=30 Jul 09 19:21:26.666922 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:26.666894 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" Jul 09 19:21:26.747373 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:26.747281 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68847\" (UniqueName: \"kubernetes.io/projected/1da73102-a001-40ed-a38f-7c2ba7318644-kube-api-access-68847\") pod \"1da73102-a001-40ed-a38f-7c2ba7318644\" (UID: \"1da73102-a001-40ed-a38f-7c2ba7318644\") " Jul 09 19:21:26.747373 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:26.747321 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1da73102-a001-40ed-a38f-7c2ba7318644-openshift-service-ca\") pod \"1da73102-a001-40ed-a38f-7c2ba7318644\" (UID: \"1da73102-a001-40ed-a38f-7c2ba7318644\") " Jul 09 19:21:26.747373 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:26.747361 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1da73102-a001-40ed-a38f-7c2ba7318644-maas-api-tls\") pod \"1da73102-a001-40ed-a38f-7c2ba7318644\" (UID: \"1da73102-a001-40ed-a38f-7c2ba7318644\") " Jul 09 19:21:26.747854 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:26.747820 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1da73102-a001-40ed-a38f-7c2ba7318644-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1da73102-a001-40ed-a38f-7c2ba7318644" (UID: "1da73102-a001-40ed-a38f-7c2ba7318644"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:21:26.749631 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:26.749596 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1da73102-a001-40ed-a38f-7c2ba7318644-kube-api-access-68847" (OuterVolumeSpecName: "kube-api-access-68847") pod "1da73102-a001-40ed-a38f-7c2ba7318644" (UID: "1da73102-a001-40ed-a38f-7c2ba7318644"). InnerVolumeSpecName "kube-api-access-68847". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:21:26.749631 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:26.749613 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1da73102-a001-40ed-a38f-7c2ba7318644-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1da73102-a001-40ed-a38f-7c2ba7318644" (UID: "1da73102-a001-40ed-a38f-7c2ba7318644"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:21:26.848045 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:26.848016 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68847\" (UniqueName: \"kubernetes.io/projected/1da73102-a001-40ed-a38f-7c2ba7318644-kube-api-access-68847\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:21:26.848045 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:26.848043 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1da73102-a001-40ed-a38f-7c2ba7318644-openshift-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:21:26.848045 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:26.848054 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1da73102-a001-40ed-a38f-7c2ba7318644-maas-api-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:21:27.002751 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:27.002658 2585 generic.go:356] "Generic (PLEG): container finished" podID="1da73102-a001-40ed-a38f-7c2ba7318644" containerID="d28f7b77df090a183752841a8dfd76a599087c34a6a12db69645e14f80328952" exitCode=0 Jul 09 19:21:27.002751 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:27.002709 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" Jul 09 19:21:27.002751 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:27.002724 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" event={"ID":"1da73102-a001-40ed-a38f-7c2ba7318644","Type":"ContainerDied","Data":"d28f7b77df090a183752841a8dfd76a599087c34a6a12db69645e14f80328952"} Jul 09 19:21:27.003021 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:27.002782 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444" event={"ID":"1da73102-a001-40ed-a38f-7c2ba7318644","Type":"ContainerDied","Data":"aa48c62f4d69a9bf403886f731200879eec4321726c4c7f293f790e6462bd515"} Jul 09 19:21:27.003021 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:27.002805 2585 scope.go:117] "RemoveContainer" containerID="d28f7b77df090a183752841a8dfd76a599087c34a6a12db69645e14f80328952" Jul 09 19:21:27.012305 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:27.012282 2585 scope.go:117] "RemoveContainer" containerID="d28f7b77df090a183752841a8dfd76a599087c34a6a12db69645e14f80328952" Jul 09 19:21:27.012607 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:21:27.012566 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d28f7b77df090a183752841a8dfd76a599087c34a6a12db69645e14f80328952\": container with ID starting with d28f7b77df090a183752841a8dfd76a599087c34a6a12db69645e14f80328952 not found: ID does not exist" containerID="d28f7b77df090a183752841a8dfd76a599087c34a6a12db69645e14f80328952" Jul 09 19:21:27.012668 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:27.012620 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d28f7b77df090a183752841a8dfd76a599087c34a6a12db69645e14f80328952"} err="failed to get container status \"d28f7b77df090a183752841a8dfd76a599087c34a6a12db69645e14f80328952\": rpc error: code = NotFound desc = could not find container \"d28f7b77df090a183752841a8dfd76a599087c34a6a12db69645e14f80328952\": container with ID starting with d28f7b77df090a183752841a8dfd76a599087c34a6a12db69645e14f80328952 not found: ID does not exist" Jul 09 19:21:27.035903 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:27.035871 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444"] Jul 09 19:21:27.054412 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:27.054387 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2f44fe-799777c86f-jm444"] Jul 09 19:21:27.843888 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:27.843848 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1da73102-a001-40ed-a38f-7c2ba7318644" path="/var/lib/kubelet/pods/1da73102-a001-40ed-a38f-7c2ba7318644/volumes" Jul 09 19:21:33.911287 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:33.911250 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz"] Jul 09 19:21:33.911740 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:33.911458 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" podUID="2269e98a-f421-43e2-b448-97458a1ecd49" containerName="maas-api" containerID="cri-o://1552290fe625d227586b5a939243dc9c451b1d95fbdbebf0a6cd77eb5fe77f50" gracePeriod=30 Jul 09 19:21:34.036203 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:34.036173 2585 generic.go:356] "Generic (PLEG): container finished" podID="2269e98a-f421-43e2-b448-97458a1ecd49" containerID="1552290fe625d227586b5a939243dc9c451b1d95fbdbebf0a6cd77eb5fe77f50" exitCode=0 Jul 09 19:21:34.036203 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:34.036214 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" event={"ID":"2269e98a-f421-43e2-b448-97458a1ecd49","Type":"ContainerDied","Data":"1552290fe625d227586b5a939243dc9c451b1d95fbdbebf0a6cd77eb5fe77f50"} Jul 09 19:21:34.160249 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:34.160225 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:34.317316 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:34.317275 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2269e98a-f421-43e2-b448-97458a1ecd49-openshift-service-ca\") pod \"2269e98a-f421-43e2-b448-97458a1ecd49\" (UID: \"2269e98a-f421-43e2-b448-97458a1ecd49\") " Jul 09 19:21:34.317524 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:34.317333 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bz28\" (UniqueName: \"kubernetes.io/projected/2269e98a-f421-43e2-b448-97458a1ecd49-kube-api-access-7bz28\") pod \"2269e98a-f421-43e2-b448-97458a1ecd49\" (UID: \"2269e98a-f421-43e2-b448-97458a1ecd49\") " Jul 09 19:21:34.317524 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:34.317402 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2269e98a-f421-43e2-b448-97458a1ecd49-maas-api-tls\") pod \"2269e98a-f421-43e2-b448-97458a1ecd49\" (UID: \"2269e98a-f421-43e2-b448-97458a1ecd49\") " Jul 09 19:21:34.317757 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:34.317730 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2269e98a-f421-43e2-b448-97458a1ecd49-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2269e98a-f421-43e2-b448-97458a1ecd49" (UID: "2269e98a-f421-43e2-b448-97458a1ecd49"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:21:34.319692 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:34.319659 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2269e98a-f421-43e2-b448-97458a1ecd49-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2269e98a-f421-43e2-b448-97458a1ecd49" (UID: "2269e98a-f421-43e2-b448-97458a1ecd49"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:21:34.319692 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:34.319683 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2269e98a-f421-43e2-b448-97458a1ecd49-kube-api-access-7bz28" (OuterVolumeSpecName: "kube-api-access-7bz28") pod "2269e98a-f421-43e2-b448-97458a1ecd49" (UID: "2269e98a-f421-43e2-b448-97458a1ecd49"). InnerVolumeSpecName "kube-api-access-7bz28". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:21:34.418689 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:34.418655 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2269e98a-f421-43e2-b448-97458a1ecd49-maas-api-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:21:34.418689 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:34.418686 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2269e98a-f421-43e2-b448-97458a1ecd49-openshift-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:21:34.418689 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:34.418696 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bz28\" (UniqueName: \"kubernetes.io/projected/2269e98a-f421-43e2-b448-97458a1ecd49-kube-api-access-7bz28\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:21:35.041884 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:35.041858 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" Jul 09 19:21:35.042315 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:35.041895 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz" event={"ID":"2269e98a-f421-43e2-b448-97458a1ecd49","Type":"ContainerDied","Data":"364bd50b06bac4efc70c52ae102d751ad3b21417838ff574311b957d3648fa2e"} Jul 09 19:21:35.042315 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:35.041954 2585 scope.go:117] "RemoveContainer" containerID="1552290fe625d227586b5a939243dc9c451b1d95fbdbebf0a6cd77eb5fe77f50" Jul 09 19:21:35.065656 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:35.065628 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz"] Jul 09 19:21:35.070862 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:35.070838 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0bfb2e-59f57f6458-vz8kz"] Jul 09 19:21:35.842419 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:35.842381 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2269e98a-f421-43e2-b448-97458a1ecd49" path="/var/lib/kubelet/pods/2269e98a-f421-43e2-b448-97458a1ecd49/volumes" Jul 09 19:21:45.091163 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.091073 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r"] Jul 09 19:21:45.091627 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.091602 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da73102-a001-40ed-a38f-7c2ba7318644" containerName="maas-api" Jul 09 19:21:45.091627 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.091620 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da73102-a001-40ed-a38f-7c2ba7318644" containerName="maas-api" Jul 09 19:21:45.091757 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.091660 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2269e98a-f421-43e2-b448-97458a1ecd49" containerName="maas-api" Jul 09 19:21:45.091757 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.091668 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2269e98a-f421-43e2-b448-97458a1ecd49" containerName="maas-api" Jul 09 19:21:45.091854 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.091766 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2269e98a-f421-43e2-b448-97458a1ecd49" containerName="maas-api" Jul 09 19:21:45.091854 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.091784 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da73102-a001-40ed-a38f-7c2ba7318644" containerName="maas-api" Jul 09 19:21:45.096376 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.096352 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" Jul 09 19:21:45.099011 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.098986 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-18e258-serving-cert\"" Jul 09 19:21:45.105302 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.105273 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r"] Jul 09 19:21:45.216349 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.216310 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg5v7\" (UniqueName: \"kubernetes.io/projected/05c5743e-67d3-431f-a1fe-fb68eef471f0-kube-api-access-qg5v7\") pod \"maas-api-e2e-shared-a-18e258-cdc89c584-gt42r\" (UID: \"05c5743e-67d3-431f-a1fe-fb68eef471f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" Jul 09 19:21:45.216349 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.216353 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05c5743e-67d3-431f-a1fe-fb68eef471f0-maas-api-tls\") pod \"maas-api-e2e-shared-a-18e258-cdc89c584-gt42r\" (UID: \"05c5743e-67d3-431f-a1fe-fb68eef471f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" Jul 09 19:21:45.216678 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.216481 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05c5743e-67d3-431f-a1fe-fb68eef471f0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-18e258-cdc89c584-gt42r\" (UID: \"05c5743e-67d3-431f-a1fe-fb68eef471f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" Jul 09 19:21:45.317728 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.317697 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05c5743e-67d3-431f-a1fe-fb68eef471f0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-18e258-cdc89c584-gt42r\" (UID: \"05c5743e-67d3-431f-a1fe-fb68eef471f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" Jul 09 19:21:45.317903 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.317755 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg5v7\" (UniqueName: \"kubernetes.io/projected/05c5743e-67d3-431f-a1fe-fb68eef471f0-kube-api-access-qg5v7\") pod \"maas-api-e2e-shared-a-18e258-cdc89c584-gt42r\" (UID: \"05c5743e-67d3-431f-a1fe-fb68eef471f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" Jul 09 19:21:45.317903 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.317780 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05c5743e-67d3-431f-a1fe-fb68eef471f0-maas-api-tls\") pod \"maas-api-e2e-shared-a-18e258-cdc89c584-gt42r\" (UID: \"05c5743e-67d3-431f-a1fe-fb68eef471f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" Jul 09 19:21:45.318343 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.318324 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05c5743e-67d3-431f-a1fe-fb68eef471f0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-18e258-cdc89c584-gt42r\" (UID: \"05c5743e-67d3-431f-a1fe-fb68eef471f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" Jul 09 19:21:45.320294 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.320275 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05c5743e-67d3-431f-a1fe-fb68eef471f0-maas-api-tls\") pod \"maas-api-e2e-shared-a-18e258-cdc89c584-gt42r\" (UID: \"05c5743e-67d3-431f-a1fe-fb68eef471f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" Jul 09 19:21:45.328320 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.328295 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg5v7\" (UniqueName: \"kubernetes.io/projected/05c5743e-67d3-431f-a1fe-fb68eef471f0-kube-api-access-qg5v7\") pod \"maas-api-e2e-shared-a-18e258-cdc89c584-gt42r\" (UID: \"05c5743e-67d3-431f-a1fe-fb68eef471f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" Jul 09 19:21:45.408378 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.408301 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" Jul 09 19:21:45.553967 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:45.553945 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r"] Jul 09 19:21:45.556004 ip-10-0-143-175 kubenswrapper[2585]: W0709 19:21:45.555937 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05c5743e_67d3_431f_a1fe_fb68eef471f0.slice/crio-5b78b4e67d1ce34ef000518add5424ecd71587303a5e4affbbf23242e4cb4540 WatchSource:0}: Error finding container 5b78b4e67d1ce34ef000518add5424ecd71587303a5e4affbbf23242e4cb4540: Status 404 returned error can't find the container with id 5b78b4e67d1ce34ef000518add5424ecd71587303a5e4affbbf23242e4cb4540 Jul 09 19:21:46.095326 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:46.095286 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" event={"ID":"05c5743e-67d3-431f-a1fe-fb68eef471f0","Type":"ContainerStarted","Data":"d3520397ed26ee6c7bd54342380c9d682e43e0d5defc0b064dd34af420742c9e"} Jul 09 19:21:46.095819 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:46.095345 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" Jul 09 19:21:46.095819 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:46.095360 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" event={"ID":"05c5743e-67d3-431f-a1fe-fb68eef471f0","Type":"ContainerStarted","Data":"5b78b4e67d1ce34ef000518add5424ecd71587303a5e4affbbf23242e4cb4540"} Jul 09 19:21:46.116797 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:46.116752 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" podStartSLOduration=0.784630708 podStartE2EDuration="1.116737044s" totalImagesPullingTime="332.106336ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:21:46.11533343 +0000 UTC m=+2422.859316536" watchObservedRunningTime="2026-07-09 19:21:46.116737044 +0000 UTC m=+2422.860720158" Jul 09 19:21:52.106140 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:52.106111 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" Jul 09 19:21:53.024344 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:53.024306 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82"] Jul 09 19:21:53.028458 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:53.028437 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:21:53.031330 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:53.031306 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-2ac82b-serving-cert\"" Jul 09 19:21:53.039030 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:53.039004 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82"] Jul 09 19:21:53.085962 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:53.085933 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/974682bf-73d1-4c7c-92ca-f4c670184b68-maas-api-tls\") pod \"maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82\" (UID: \"974682bf-73d1-4c7c-92ca-f4c670184b68\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:21:53.085962 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:53.085975 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbn7w\" (UniqueName: \"kubernetes.io/projected/974682bf-73d1-4c7c-92ca-f4c670184b68-kube-api-access-vbn7w\") pod \"maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82\" (UID: \"974682bf-73d1-4c7c-92ca-f4c670184b68\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:21:53.086202 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:53.086061 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/974682bf-73d1-4c7c-92ca-f4c670184b68-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82\" (UID: \"974682bf-73d1-4c7c-92ca-f4c670184b68\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:21:53.186438 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:53.186405 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/974682bf-73d1-4c7c-92ca-f4c670184b68-maas-api-tls\") pod \"maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82\" (UID: \"974682bf-73d1-4c7c-92ca-f4c670184b68\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:21:53.186438 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:53.186439 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbn7w\" (UniqueName: \"kubernetes.io/projected/974682bf-73d1-4c7c-92ca-f4c670184b68-kube-api-access-vbn7w\") pod \"maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82\" (UID: \"974682bf-73d1-4c7c-92ca-f4c670184b68\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:21:53.186976 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:53.186474 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/974682bf-73d1-4c7c-92ca-f4c670184b68-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82\" (UID: \"974682bf-73d1-4c7c-92ca-f4c670184b68\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:21:53.186976 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:21:53.186561 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-serving-cert: secret "maas-api-e2e-shared-b-2ac82b-serving-cert" not found Jul 09 19:21:53.186976 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:21:53.186662 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/974682bf-73d1-4c7c-92ca-f4c670184b68-maas-api-tls podName:974682bf-73d1-4c7c-92ca-f4c670184b68 nodeName:}" failed. No retries permitted until 2026-07-09 19:21:53.686645317 +0000 UTC m=+2430.430628408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/974682bf-73d1-4c7c-92ca-f4c670184b68-maas-api-tls") pod "maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" (UID: "974682bf-73d1-4c7c-92ca-f4c670184b68") : secret "maas-api-e2e-shared-b-2ac82b-serving-cert" not found Jul 09 19:21:53.187153 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:53.187131 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/974682bf-73d1-4c7c-92ca-f4c670184b68-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82\" (UID: \"974682bf-73d1-4c7c-92ca-f4c670184b68\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:21:53.196622 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:53.196578 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbn7w\" (UniqueName: \"kubernetes.io/projected/974682bf-73d1-4c7c-92ca-f4c670184b68-kube-api-access-vbn7w\") pod \"maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82\" (UID: \"974682bf-73d1-4c7c-92ca-f4c670184b68\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:21:53.690401 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:53.690365 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/974682bf-73d1-4c7c-92ca-f4c670184b68-maas-api-tls\") pod \"maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82\" (UID: \"974682bf-73d1-4c7c-92ca-f4c670184b68\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:21:53.692990 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:53.692964 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/974682bf-73d1-4c7c-92ca-f4c670184b68-maas-api-tls\") pod \"maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82\" (UID: \"974682bf-73d1-4c7c-92ca-f4c670184b68\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:21:53.942752 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:53.942667 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:21:54.087470 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:54.087444 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82"] Jul 09 19:21:54.089733 ip-10-0-143-175 kubenswrapper[2585]: W0709 19:21:54.089705 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod974682bf_73d1_4c7c_92ca_f4c670184b68.slice/crio-7732876ae22efabe7374d9a527602cb79ca4eb28b2faa34404efa7205f4662bb WatchSource:0}: Error finding container 7732876ae22efabe7374d9a527602cb79ca4eb28b2faa34404efa7205f4662bb: Status 404 returned error can't find the container with id 7732876ae22efabe7374d9a527602cb79ca4eb28b2faa34404efa7205f4662bb Jul 09 19:21:54.091094 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:54.091078 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 19:21:54.134734 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:54.134697 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" event={"ID":"974682bf-73d1-4c7c-92ca-f4c670184b68","Type":"ContainerStarted","Data":"7732876ae22efabe7374d9a527602cb79ca4eb28b2faa34404efa7205f4662bb"} Jul 09 19:21:55.141185 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:55.141149 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" event={"ID":"974682bf-73d1-4c7c-92ca-f4c670184b68","Type":"ContainerStarted","Data":"e52c01e2f2aa3696a22a2a52009c3a2495f79db863c25111e6e13aad9f2a348f"} Jul 09 19:21:55.141559 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:55.141244 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:21:55.162247 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:21:55.162192 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" podStartSLOduration=1.74864004 podStartE2EDuration="2.16217724s" totalImagesPullingTime="413.5372ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:21:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:21:55.159520245 +0000 UTC m=+2431.903503359" watchObservedRunningTime="2026-07-09 19:21:55.16217724 +0000 UTC m=+2431.906160354" Jul 09 19:22:01.151988 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:22:01.151960 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:26:24.610726 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:26:24.610604 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 19:26:24.617410 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:26:24.617388 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gzm9x_5a43cf7e-2232-4fdc-aa1d-4ac08f9e9985/ovn-acl-logging/0.log" Jul 09 19:27:39.284520 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.284490 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r"] Jul 09 19:27:39.285026 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.284795 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" podUID="05c5743e-67d3-431f-a1fe-fb68eef471f0" containerName="maas-api" containerID="cri-o://d3520397ed26ee6c7bd54342380c9d682e43e0d5defc0b064dd34af420742c9e" gracePeriod=30 Jul 09 19:27:39.532561 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.532535 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" Jul 09 19:27:39.613209 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.613129 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qg5v7\" (UniqueName: \"kubernetes.io/projected/05c5743e-67d3-431f-a1fe-fb68eef471f0-kube-api-access-qg5v7\") pod \"05c5743e-67d3-431f-a1fe-fb68eef471f0\" (UID: \"05c5743e-67d3-431f-a1fe-fb68eef471f0\") " Jul 09 19:27:39.613209 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.613168 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05c5743e-67d3-431f-a1fe-fb68eef471f0-maas-api-tls\") pod \"05c5743e-67d3-431f-a1fe-fb68eef471f0\" (UID: \"05c5743e-67d3-431f-a1fe-fb68eef471f0\") " Jul 09 19:27:39.613440 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.613227 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05c5743e-67d3-431f-a1fe-fb68eef471f0-openshift-service-ca\") pod \"05c5743e-67d3-431f-a1fe-fb68eef471f0\" (UID: \"05c5743e-67d3-431f-a1fe-fb68eef471f0\") " Jul 09 19:27:39.613656 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.613618 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05c5743e-67d3-431f-a1fe-fb68eef471f0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "05c5743e-67d3-431f-a1fe-fb68eef471f0" (UID: "05c5743e-67d3-431f-a1fe-fb68eef471f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:27:39.615488 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.615456 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05c5743e-67d3-431f-a1fe-fb68eef471f0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "05c5743e-67d3-431f-a1fe-fb68eef471f0" (UID: "05c5743e-67d3-431f-a1fe-fb68eef471f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:27:39.615488 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.615477 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05c5743e-67d3-431f-a1fe-fb68eef471f0-kube-api-access-qg5v7" (OuterVolumeSpecName: "kube-api-access-qg5v7") pod "05c5743e-67d3-431f-a1fe-fb68eef471f0" (UID: "05c5743e-67d3-431f-a1fe-fb68eef471f0"). InnerVolumeSpecName "kube-api-access-qg5v7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:27:39.714249 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.714217 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qg5v7\" (UniqueName: \"kubernetes.io/projected/05c5743e-67d3-431f-a1fe-fb68eef471f0-kube-api-access-qg5v7\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:27:39.714249 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.714244 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05c5743e-67d3-431f-a1fe-fb68eef471f0-maas-api-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:27:39.714415 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.714257 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05c5743e-67d3-431f-a1fe-fb68eef471f0-openshift-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:27:39.732189 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.732157 2585 generic.go:356] "Generic (PLEG): container finished" podID="05c5743e-67d3-431f-a1fe-fb68eef471f0" containerID="d3520397ed26ee6c7bd54342380c9d682e43e0d5defc0b064dd34af420742c9e" exitCode=0 Jul 09 19:27:39.732296 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.732208 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" Jul 09 19:27:39.732296 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.732245 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" event={"ID":"05c5743e-67d3-431f-a1fe-fb68eef471f0","Type":"ContainerDied","Data":"d3520397ed26ee6c7bd54342380c9d682e43e0d5defc0b064dd34af420742c9e"} Jul 09 19:27:39.732371 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.732306 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r" event={"ID":"05c5743e-67d3-431f-a1fe-fb68eef471f0","Type":"ContainerDied","Data":"5b78b4e67d1ce34ef000518add5424ecd71587303a5e4affbbf23242e4cb4540"} Jul 09 19:27:39.732371 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.732332 2585 scope.go:117] "RemoveContainer" containerID="d3520397ed26ee6c7bd54342380c9d682e43e0d5defc0b064dd34af420742c9e" Jul 09 19:27:39.741321 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.741300 2585 scope.go:117] "RemoveContainer" containerID="d3520397ed26ee6c7bd54342380c9d682e43e0d5defc0b064dd34af420742c9e" Jul 09 19:27:39.741615 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:27:39.741574 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3520397ed26ee6c7bd54342380c9d682e43e0d5defc0b064dd34af420742c9e\": container with ID starting with d3520397ed26ee6c7bd54342380c9d682e43e0d5defc0b064dd34af420742c9e not found: ID does not exist" containerID="d3520397ed26ee6c7bd54342380c9d682e43e0d5defc0b064dd34af420742c9e" Jul 09 19:27:39.741697 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.741621 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3520397ed26ee6c7bd54342380c9d682e43e0d5defc0b064dd34af420742c9e"} err="failed to get container status \"d3520397ed26ee6c7bd54342380c9d682e43e0d5defc0b064dd34af420742c9e\": rpc error: code = NotFound desc = could not find container \"d3520397ed26ee6c7bd54342380c9d682e43e0d5defc0b064dd34af420742c9e\": container with ID starting with d3520397ed26ee6c7bd54342380c9d682e43e0d5defc0b064dd34af420742c9e not found: ID does not exist" Jul 09 19:27:39.755800 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.755775 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r"] Jul 09 19:27:39.760534 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.760514 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-18e258-cdc89c584-gt42r"] Jul 09 19:27:39.843631 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:39.843571 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05c5743e-67d3-431f-a1fe-fb68eef471f0" path="/var/lib/kubelet/pods/05c5743e-67d3-431f-a1fe-fb68eef471f0/volumes" Jul 09 19:27:47.120034 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.119956 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82"] Jul 09 19:27:47.120406 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.120232 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" podUID="974682bf-73d1-4c7c-92ca-f4c670184b68" containerName="maas-api" containerID="cri-o://e52c01e2f2aa3696a22a2a52009c3a2495f79db863c25111e6e13aad9f2a348f" gracePeriod=30 Jul 09 19:27:47.368761 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.368737 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:27:47.484964 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.484935 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbn7w\" (UniqueName: \"kubernetes.io/projected/974682bf-73d1-4c7c-92ca-f4c670184b68-kube-api-access-vbn7w\") pod \"974682bf-73d1-4c7c-92ca-f4c670184b68\" (UID: \"974682bf-73d1-4c7c-92ca-f4c670184b68\") " Jul 09 19:27:47.485159 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.484982 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/974682bf-73d1-4c7c-92ca-f4c670184b68-maas-api-tls\") pod \"974682bf-73d1-4c7c-92ca-f4c670184b68\" (UID: \"974682bf-73d1-4c7c-92ca-f4c670184b68\") " Jul 09 19:27:47.485159 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.485004 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/974682bf-73d1-4c7c-92ca-f4c670184b68-openshift-service-ca\") pod \"974682bf-73d1-4c7c-92ca-f4c670184b68\" (UID: \"974682bf-73d1-4c7c-92ca-f4c670184b68\") " Jul 09 19:27:47.485438 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.485412 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/974682bf-73d1-4c7c-92ca-f4c670184b68-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "974682bf-73d1-4c7c-92ca-f4c670184b68" (UID: "974682bf-73d1-4c7c-92ca-f4c670184b68"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:27:47.487323 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.487296 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/974682bf-73d1-4c7c-92ca-f4c670184b68-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "974682bf-73d1-4c7c-92ca-f4c670184b68" (UID: "974682bf-73d1-4c7c-92ca-f4c670184b68"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:27:47.487323 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.487305 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/974682bf-73d1-4c7c-92ca-f4c670184b68-kube-api-access-vbn7w" (OuterVolumeSpecName: "kube-api-access-vbn7w") pod "974682bf-73d1-4c7c-92ca-f4c670184b68" (UID: "974682bf-73d1-4c7c-92ca-f4c670184b68"). InnerVolumeSpecName "kube-api-access-vbn7w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:27:47.585945 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.585914 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbn7w\" (UniqueName: \"kubernetes.io/projected/974682bf-73d1-4c7c-92ca-f4c670184b68-kube-api-access-vbn7w\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:27:47.585945 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.585941 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/974682bf-73d1-4c7c-92ca-f4c670184b68-maas-api-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:27:47.585945 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.585951 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/974682bf-73d1-4c7c-92ca-f4c670184b68-openshift-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Jul 09 19:27:47.772543 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.772458 2585 generic.go:356] "Generic (PLEG): container finished" podID="974682bf-73d1-4c7c-92ca-f4c670184b68" containerID="e52c01e2f2aa3696a22a2a52009c3a2495f79db863c25111e6e13aad9f2a348f" exitCode=0 Jul 09 19:27:47.772543 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.772510 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" Jul 09 19:27:47.772755 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.772552 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" event={"ID":"974682bf-73d1-4c7c-92ca-f4c670184b68","Type":"ContainerDied","Data":"e52c01e2f2aa3696a22a2a52009c3a2495f79db863c25111e6e13aad9f2a348f"} Jul 09 19:27:47.772755 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.772636 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82" event={"ID":"974682bf-73d1-4c7c-92ca-f4c670184b68","Type":"ContainerDied","Data":"7732876ae22efabe7374d9a527602cb79ca4eb28b2faa34404efa7205f4662bb"} Jul 09 19:27:47.772755 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.772668 2585 scope.go:117] "RemoveContainer" containerID="e52c01e2f2aa3696a22a2a52009c3a2495f79db863c25111e6e13aad9f2a348f" Jul 09 19:27:47.781886 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.781866 2585 scope.go:117] "RemoveContainer" containerID="e52c01e2f2aa3696a22a2a52009c3a2495f79db863c25111e6e13aad9f2a348f" Jul 09 19:27:47.782145 ip-10-0-143-175 kubenswrapper[2585]: E0709 19:27:47.782126 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e52c01e2f2aa3696a22a2a52009c3a2495f79db863c25111e6e13aad9f2a348f\": container with ID starting with e52c01e2f2aa3696a22a2a52009c3a2495f79db863c25111e6e13aad9f2a348f not found: ID does not exist" containerID="e52c01e2f2aa3696a22a2a52009c3a2495f79db863c25111e6e13aad9f2a348f" Jul 09 19:27:47.782210 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.782158 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e52c01e2f2aa3696a22a2a52009c3a2495f79db863c25111e6e13aad9f2a348f"} err="failed to get container status \"e52c01e2f2aa3696a22a2a52009c3a2495f79db863c25111e6e13aad9f2a348f\": rpc error: code = NotFound desc = could not find container \"e52c01e2f2aa3696a22a2a52009c3a2495f79db863c25111e6e13aad9f2a348f\": container with ID starting with e52c01e2f2aa3696a22a2a52009c3a2495f79db863c25111e6e13aad9f2a348f not found: ID does not exist" Jul 09 19:27:47.795515 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.795483 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82"] Jul 09 19:27:47.797387 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.797363 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2ac82b-7855c59bc4-hvv82"] Jul 09 19:27:47.842429 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:47.842396 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="974682bf-73d1-4c7c-92ca-f4c670184b68" path="/var/lib/kubelet/pods/974682bf-73d1-4c7c-92ca-f4c670184b68/volumes" Jul 09 19:27:54.325134 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:54.325091 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55cf685454-bzw97_43f096dd-39ad-4ca5-a8c0-315345407e32/authorino/0.log" Jul 09 19:27:59.570033 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:59.569976 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f5cd9bc7b-5tf2w_ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf/manager/0.log" Jul 09 19:27:59.941841 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:59.941812 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fbd77dc5f-244nc_17c4b9fd-a28d-4960-aaa1-b336aa6497ec/copy-manifests/0.log" Jul 09 19:27:59.951274 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:27:59.951245 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fbd77dc5f-244nc_17c4b9fd-a28d-4960-aaa1-b336aa6497ec/manager/0.log" Jul 09 19:28:00.570834 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:00.570798 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-d89b88c4c-ccnjk_235c900d-b372-41f9-bf33-0f1a5bf90e63/maas-api/0.log" Jul 09 19:28:00.724325 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:00.724289 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-r5f7q_d3ca36a1-0fae-4899-adc5-73b5d165b972/postgres/0.log" Jul 09 19:28:02.052147 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:02.052119 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55cf685454-bzw97_43f096dd-39ad-4ca5-a8c0-315345407e32/authorino/0.log" Jul 09 19:28:03.506123 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:03.506090 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74785d646-htfxm_fef65122-b577-4ccd-a8a6-5cebf06b7ab2/kube-auth-proxy/0.log" Jul 09 19:28:03.862872 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:03.862799 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-v2qj2_c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d/payload-pre-processing/0.log" Jul 09 19:28:03.981032 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:03.981008 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bh4h4_431f617b-dd44-4b8d-80f8-ee8cfc290998/payload-processing/0.log" Jul 09 19:28:04.564925 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:04.564899 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-dhmmc_851b617b-168b-41b4-96db-8ca8cf61257d/main/0.log" Jul 09 19:28:04.799255 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:04.799219 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db92nqw_4db54f1b-62a6-4145-b525-cd84723a0e8e/main/0.log" Jul 09 19:28:11.609445 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:11.609094 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8mts9_fda064a0-8a88-462c-9891-47a412d9c0f4/global-pull-secret-syncer/0.log" Jul 09 19:28:11.779561 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:11.779528 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pdlv2_c472afa5-1c0f-4663-9d81-12f9050de2a7/konnectivity-agent/0.log" Jul 09 19:28:11.868807 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:11.868731 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-175.ec2.internal_f1cebde9eeebb426c6f5a392a68065db/haproxy/0.log" Jul 09 19:28:16.360213 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:16.360182 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55cf685454-bzw97_43f096dd-39ad-4ca5-a8c0-315345407e32/authorino/0.log" Jul 09 19:28:18.762794 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:18.762758 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-bs9hl_19620443-1eb7-4610-b9fb-11aa31300c89/cluster-monitoring-operator/0.log" Jul 09 19:28:18.799041 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:18.798958 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-bvdkj_99dccb5e-972b-49fe-9fff-8204eabb4335/kube-state-metrics/0.log" Jul 09 19:28:18.834343 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:18.834312 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-bvdkj_99dccb5e-972b-49fe-9fff-8204eabb4335/kube-rbac-proxy-main/0.log" Jul 09 19:28:18.885861 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:18.885829 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-bvdkj_99dccb5e-972b-49fe-9fff-8204eabb4335/kube-rbac-proxy-self/0.log" Jul 09 19:28:18.992876 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:18.992845 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-sqg4q_613b7cd8-3e1c-4ab1-805f-769e799a8a27/monitoring-plugin/0.log" Jul 09 19:28:19.050054 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:19.049977 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-67vv4_9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6/node-exporter/0.log" Jul 09 19:28:19.092534 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:19.092505 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-67vv4_9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6/kube-rbac-proxy/0.log" Jul 09 19:28:19.126048 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:19.126018 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-67vv4_9b5d8f55-7a0e-4bbc-ac2e-2914a88d44e6/init-textfile/0.log" Jul 09 19:28:19.452149 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:19.452122 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_50021af2-6813-4fb0-b11a-d70fdecdc0ed/prometheus/0.log" Jul 09 19:28:19.481719 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:19.481687 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_50021af2-6813-4fb0-b11a-d70fdecdc0ed/config-reloader/0.log" Jul 09 19:28:19.513173 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:19.513143 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_50021af2-6813-4fb0-b11a-d70fdecdc0ed/thanos-sidecar/0.log" Jul 09 19:28:19.541470 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:19.541433 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_50021af2-6813-4fb0-b11a-d70fdecdc0ed/kube-rbac-proxy-web/0.log" Jul 09 19:28:19.570608 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:19.570556 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_50021af2-6813-4fb0-b11a-d70fdecdc0ed/kube-rbac-proxy/0.log" Jul 09 19:28:19.597687 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:19.597660 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_50021af2-6813-4fb0-b11a-d70fdecdc0ed/kube-rbac-proxy-thanos/0.log" Jul 09 19:28:19.625100 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:19.625079 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_50021af2-6813-4fb0-b11a-d70fdecdc0ed/init-config-reloader/0.log" Jul 09 19:28:21.148784 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.148754 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-qpgdh_383d00ba-23bb-45b1-a968-71b5dda948c5/networking-console-plugin/0.log" Jul 09 19:28:21.274094 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.274060 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2"] Jul 09 19:28:21.274513 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.274496 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05c5743e-67d3-431f-a1fe-fb68eef471f0" containerName="maas-api" Jul 09 19:28:21.274629 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.274516 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="05c5743e-67d3-431f-a1fe-fb68eef471f0" containerName="maas-api" Jul 09 19:28:21.274629 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.274539 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="974682bf-73d1-4c7c-92ca-f4c670184b68" containerName="maas-api" Jul 09 19:28:21.274629 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.274547 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="974682bf-73d1-4c7c-92ca-f4c670184b68" containerName="maas-api" Jul 09 19:28:21.274789 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.274664 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="05c5743e-67d3-431f-a1fe-fb68eef471f0" containerName="maas-api" Jul 09 19:28:21.274789 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.274679 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="974682bf-73d1-4c7c-92ca-f4c670184b68" containerName="maas-api" Jul 09 19:28:21.277904 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.277881 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.280666 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.280641 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b6jw8\"/\"openshift-service-ca.crt\"" Jul 09 19:28:21.281639 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.281624 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b6jw8\"/\"kube-root-ca.crt\"" Jul 09 19:28:21.281639 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.281631 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b6jw8\"/\"default-dockercfg-fk46d\"" Jul 09 19:28:21.288705 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.288683 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2"] Jul 09 19:28:21.380660 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.380626 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fbf1b409-0625-4beb-953d-f4cd16738b5b-lib-modules\") pod \"perf-node-gather-daemonset-khrk2\" (UID: \"fbf1b409-0625-4beb-953d-f4cd16738b5b\") " pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.380660 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.380664 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qml2\" (UniqueName: \"kubernetes.io/projected/fbf1b409-0625-4beb-953d-f4cd16738b5b-kube-api-access-2qml2\") pod \"perf-node-gather-daemonset-khrk2\" (UID: \"fbf1b409-0625-4beb-953d-f4cd16738b5b\") " pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.380906 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.380744 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbf1b409-0625-4beb-953d-f4cd16738b5b-sys\") pod \"perf-node-gather-daemonset-khrk2\" (UID: \"fbf1b409-0625-4beb-953d-f4cd16738b5b\") " pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.380906 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.380761 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fbf1b409-0625-4beb-953d-f4cd16738b5b-podres\") pod \"perf-node-gather-daemonset-khrk2\" (UID: \"fbf1b409-0625-4beb-953d-f4cd16738b5b\") " pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.380906 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.380843 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fbf1b409-0625-4beb-953d-f4cd16738b5b-proc\") pod \"perf-node-gather-daemonset-khrk2\" (UID: \"fbf1b409-0625-4beb-953d-f4cd16738b5b\") " pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.481714 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.481678 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fbf1b409-0625-4beb-953d-f4cd16738b5b-lib-modules\") pod \"perf-node-gather-daemonset-khrk2\" (UID: \"fbf1b409-0625-4beb-953d-f4cd16738b5b\") " pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.481920 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.481719 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qml2\" (UniqueName: \"kubernetes.io/projected/fbf1b409-0625-4beb-953d-f4cd16738b5b-kube-api-access-2qml2\") pod \"perf-node-gather-daemonset-khrk2\" (UID: \"fbf1b409-0625-4beb-953d-f4cd16738b5b\") " pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.481920 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.481820 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbf1b409-0625-4beb-953d-f4cd16738b5b-sys\") pod \"perf-node-gather-daemonset-khrk2\" (UID: \"fbf1b409-0625-4beb-953d-f4cd16738b5b\") " pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.481920 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.481843 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fbf1b409-0625-4beb-953d-f4cd16738b5b-podres\") pod \"perf-node-gather-daemonset-khrk2\" (UID: \"fbf1b409-0625-4beb-953d-f4cd16738b5b\") " pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.481920 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.481874 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fbf1b409-0625-4beb-953d-f4cd16738b5b-proc\") pod \"perf-node-gather-daemonset-khrk2\" (UID: \"fbf1b409-0625-4beb-953d-f4cd16738b5b\") " pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.481920 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.481876 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fbf1b409-0625-4beb-953d-f4cd16738b5b-lib-modules\") pod \"perf-node-gather-daemonset-khrk2\" (UID: \"fbf1b409-0625-4beb-953d-f4cd16738b5b\") " pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.481920 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.481888 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbf1b409-0625-4beb-953d-f4cd16738b5b-sys\") pod \"perf-node-gather-daemonset-khrk2\" (UID: \"fbf1b409-0625-4beb-953d-f4cd16738b5b\") " pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.482155 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.481946 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fbf1b409-0625-4beb-953d-f4cd16738b5b-proc\") pod \"perf-node-gather-daemonset-khrk2\" (UID: \"fbf1b409-0625-4beb-953d-f4cd16738b5b\") " pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.482155 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.481998 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fbf1b409-0625-4beb-953d-f4cd16738b5b-podres\") pod \"perf-node-gather-daemonset-khrk2\" (UID: \"fbf1b409-0625-4beb-953d-f4cd16738b5b\") " pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.492030 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.492009 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qml2\" (UniqueName: \"kubernetes.io/projected/fbf1b409-0625-4beb-953d-f4cd16738b5b-kube-api-access-2qml2\") pod \"perf-node-gather-daemonset-khrk2\" (UID: \"fbf1b409-0625-4beb-953d-f4cd16738b5b\") " pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.588942 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.588907 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.725802 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.725771 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2"] Jul 09 19:28:21.728311 ip-10-0-143-175 kubenswrapper[2585]: W0709 19:28:21.728285 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfbf1b409_0625_4beb_953d_f4cd16738b5b.slice/crio-faef6e2a709779d831c681e639d2d5445e40a90ca18d6fba844ebc3a2b0e9fdb WatchSource:0}: Error finding container faef6e2a709779d831c681e639d2d5445e40a90ca18d6fba844ebc3a2b0e9fdb: Status 404 returned error can't find the container with id faef6e2a709779d831c681e639d2d5445e40a90ca18d6fba844ebc3a2b0e9fdb Jul 09 19:28:21.948940 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.948886 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" event={"ID":"fbf1b409-0625-4beb-953d-f4cd16738b5b","Type":"ContainerStarted","Data":"ea039cffcec0ce9f5f02052741b27fc79390c878774bd1ddbad52045d80131c5"} Jul 09 19:28:21.949138 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.949084 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:21.949207 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.949140 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" event={"ID":"fbf1b409-0625-4beb-953d-f4cd16738b5b","Type":"ContainerStarted","Data":"faef6e2a709779d831c681e639d2d5445e40a90ca18d6fba844ebc3a2b0e9fdb"} Jul 09 19:28:21.968053 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:21.968003 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" podStartSLOduration=0.967987952 podStartE2EDuration="967.987952ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:28:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:28:21.966988855 +0000 UTC m=+2818.710971970" watchObservedRunningTime="2026-07-09 19:28:21.967987952 +0000 UTC m=+2818.711971068" Jul 09 19:28:22.465376 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:22.465333 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-m8nk7_68f27912-05a0-4cf9-ab28-e97d87bb5f19/download-server/0.log" Jul 09 19:28:24.175089 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:24.175049 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7xb7b_f9ae055f-9c62-4b47-b584-c1941319a5ed/dns/0.log" Jul 09 19:28:24.205617 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:24.205550 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7xb7b_f9ae055f-9c62-4b47-b584-c1941319a5ed/kube-rbac-proxy/0.log" Jul 09 19:28:24.404341 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:24.404265 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wb72f_397a8531-67a9-4a3f-ad6b-f450a14856a3/dns-node-resolver/0.log" Jul 09 19:28:25.086971 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:25.086943 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vswpc_a21694d7-e996-4434-ba98-4dbfed837bd2/node-ca/0.log" Jul 09 19:28:26.227861 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:26.227831 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74785d646-htfxm_fef65122-b577-4ccd-a8a6-5cebf06b7ab2/kube-auth-proxy/0.log" Jul 09 19:28:26.377393 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:26.377360 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-v2qj2_c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d/payload-pre-processing/0.log" Jul 09 19:28:26.409437 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:26.409408 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bh4h4_431f617b-dd44-4b8d-80f8-ee8cfc290998/payload-processing/0.log" Jul 09 19:28:27.225632 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:27.225603 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m4mmf_f882fb17-780a-4bff-a2b1-80b092a839c3/serve-healthcheck-canary/0.log" Jul 09 19:28:27.944328 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:27.944302 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r8dks_e7275e4d-1d71-4b91-9ae2-a78ef40557e7/kube-rbac-proxy/0.log" Jul 09 19:28:27.964205 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:27.964179 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b6jw8/perf-node-gather-daemonset-khrk2" Jul 09 19:28:27.972768 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:27.972740 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r8dks_e7275e4d-1d71-4b91-9ae2-a78ef40557e7/exporter/0.log" Jul 09 19:28:28.005282 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:28.005255 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r8dks_e7275e4d-1d71-4b91-9ae2-a78ef40557e7/extractor/0.log" Jul 09 19:28:30.575047 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:30.574964 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f5cd9bc7b-5tf2w_ac55a28b-cd8b-4bcf-b714-f50b2a2d0ecf/manager/0.log" Jul 09 19:28:30.709018 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:30.708993 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fbd77dc5f-244nc_17c4b9fd-a28d-4960-aaa1-b336aa6497ec/manager/0.log" Jul 09 19:28:30.744022 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:30.743997 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fbd77dc5f-244nc_17c4b9fd-a28d-4960-aaa1-b336aa6497ec/copy-manifests/0.log" Jul 09 19:28:31.782071 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:31.782037 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74785d646-htfxm_fef65122-b577-4ccd-a8a6-5cebf06b7ab2/kube-auth-proxy/0.log" Jul 09 19:28:31.882199 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:31.882164 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-v2qj2_c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d/payload-pre-processing/0.log" Jul 09 19:28:31.902326 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:31.902299 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bh4h4_431f617b-dd44-4b8d-80f8-ee8cfc290998/payload-processing/0.log" Jul 09 19:28:32.225962 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:32.225940 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-68b945dcf-jnld5_5be5c13a-6e34-4dc1-ab0e-9ebe724f91ec/manager/0.log" Jul 09 19:28:32.301372 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:32.301347 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rjjvt_b7085015-f3cd-45c6-9c9b-273914615ecb/openshift-lws-operator/0.log" Jul 09 19:28:33.316438 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:33.316406 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74785d646-htfxm_fef65122-b577-4ccd-a8a6-5cebf06b7ab2/kube-auth-proxy/0.log" Jul 09 19:28:33.429642 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:33.429610 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-v2qj2_c00f0b7e-6103-4cbb-9d62-4ae4569e0f5d/payload-pre-processing/0.log" Jul 09 19:28:33.452749 ip-10-0-143-175 kubenswrapper[2585]: I0709 19:28:33.452722 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bh4h4_431f617b-dd44-4b8d-80f8-ee8cfc290998/payload-processing/0.log"