Jun 11 21:10:26.165359 ip-10-0-140-167 systemd[1]: Starting Kubernetes Kubelet... Jun 11 21:10:26.465035 ip-10-0-140-167 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 21:10:26.465035 ip-10-0-140-167 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 11 21:10:26.465035 ip-10-0-140-167 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 21:10:26.465035 ip-10-0-140-167 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 11 21:10:26.465035 ip-10-0-140-167 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 21:10:26.466954 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.466776 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 11 21:10:26.470372 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470349 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 21:10:26.470372 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470367 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 21:10:26.470372 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470374 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 21:10:26.470372 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470378 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470383 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470388 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470392 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470396 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470400 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470404 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470407 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470410 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470414 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470418 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470422 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470426 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470430 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470434 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470445 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470449 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470453 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470457 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470461 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470465 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 21:10:26.470616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470469 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470473 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470477 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470481 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470485 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470489 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470493 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470497 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470501 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470508 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470512 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470516 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470521 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470527 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470533 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470537 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470542 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470546 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470550 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 21:10:26.471491 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470554 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470559 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470566 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470571 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470575 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470581 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470585 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470589 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470593 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470597 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470602 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470606 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470610 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470614 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470619 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470624 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470631 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470636 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470641 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 21:10:26.472387 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470646 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470651 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470656 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470661 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470666 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470670 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470675 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470680 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470685 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470691 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470696 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470701 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470706 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470711 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470715 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470720 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470724 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470730 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470735 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470739 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 21:10:26.473208 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470744 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470749 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470753 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470758 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470763 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470768 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470772 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470776 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470781 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470785 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470790 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470794 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470801 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470807 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470811 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470816 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.470820 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472014 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472043 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 21:10:26.474042 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472048 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472053 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472058 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472063 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472068 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472072 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472076 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472080 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472084 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472088 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472093 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472097 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472101 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472105 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472109 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472114 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472121 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472126 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472131 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472136 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 21:10:26.474514 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472140 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472145 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472149 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472153 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472157 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472162 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472166 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472170 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472174 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472178 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472182 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472186 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472190 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472194 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472198 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472203 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472207 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472211 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472216 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472220 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 21:10:26.475010 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472224 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472229 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472233 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472237 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472241 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472245 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472249 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472254 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472258 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472267 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472271 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472276 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472280 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472284 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472288 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472292 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472296 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472302 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472306 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472310 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 21:10:26.475581 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472314 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472318 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472323 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472329 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472335 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472339 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472344 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472349 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472354 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472360 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472364 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472369 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472373 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472377 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472381 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472385 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472389 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472393 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472396 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 21:10:26.476103 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472401 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472405 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472411 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472415 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472421 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472427 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472432 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472436 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472442 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472446 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472451 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472455 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472459 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472464 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472468 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472472 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472476 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.472480 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473129 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 21:10:26.476570 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473140 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473145 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473150 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473156 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473160 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473164 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473171 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473177 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473181 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473186 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473190 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473195 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473199 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473203 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473208 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473214 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473219 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473223 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473227 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473231 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 21:10:26.477052 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473235 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473240 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473244 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473248 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473253 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473257 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473261 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473265 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473269 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473273 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473278 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473282 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473287 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473292 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473297 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473301 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473314 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473318 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473325 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473331 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 21:10:26.477537 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473336 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473341 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473345 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473350 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473354 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473358 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473363 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473369 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473373 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473377 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473381 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473385 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473389 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473393 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473397 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473402 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473407 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473411 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473415 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 21:10:26.478063 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473418 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473422 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473425 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473429 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473433 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473437 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473441 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473444 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473448 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473452 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473464 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473469 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473473 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473477 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473480 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473484 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473488 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473492 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473498 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473502 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 21:10:26.478531 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473507 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473511 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473515 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473519 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473523 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473527 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473531 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473536 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473540 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473544 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473547 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473552 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473556 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473560 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473565 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473569 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473573 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473578 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.473582 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473687 2563 flags.go:64] FLAG: --address="0.0.0.0" Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473699 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 11 21:10:26.479040 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473714 2563 flags.go:64] FLAG: --anonymous-auth="true" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473721 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473735 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473741 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473747 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473755 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473760 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473765 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473770 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473778 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473783 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473790 2563 flags.go:64] FLAG: --cgroup-root="" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473796 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473800 2563 flags.go:64] FLAG: --client-ca-file="" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473805 2563 flags.go:64] FLAG: --cloud-provider="external" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473810 2563 flags.go:64] FLAG: --cluster-dns="[]" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473819 2563 flags.go:64] FLAG: --cluster-domain="" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473824 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473829 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473867 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473875 2563 flags.go:64] FLAG: --container-log-max-files="5" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473888 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473893 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 11 21:10:26.479557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473896 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473900 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473903 2563 flags.go:64] FLAG: --contention-profiling="false" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473907 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473910 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473914 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473917 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473922 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473926 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473929 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473932 2563 flags.go:64] FLAG: --enable-load-reader="false" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473936 2563 flags.go:64] FLAG: --enable-server="true" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473939 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473945 2563 flags.go:64] FLAG: --event-burst="100" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473949 2563 flags.go:64] FLAG: --event-qps="50" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473952 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473955 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473958 2563 flags.go:64] FLAG: --eviction-hard="" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473962 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473965 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473968 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473971 2563 flags.go:64] FLAG: --eviction-soft="" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473974 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473977 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473980 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 11 21:10:26.480152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473983 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473985 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473989 2563 flags.go:64] FLAG: --fail-swap-on="true" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473992 2563 flags.go:64] FLAG: --feature-gates="" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.473998 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474001 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474005 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474008 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474011 2563 flags.go:64] FLAG: --healthz-port="10248" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474015 2563 flags.go:64] FLAG: --help="false" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474031 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474035 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474039 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474042 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474046 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474050 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474053 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474056 2563 flags.go:64] FLAG: --image-service-endpoint="" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474059 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474062 2563 flags.go:64] FLAG: --kube-api-burst="100" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474065 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474069 2563 flags.go:64] FLAG: --kube-api-qps="50" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474072 2563 flags.go:64] FLAG: --kube-reserved="" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474075 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 11 21:10:26.481189 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474078 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474081 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474084 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474087 2563 flags.go:64] FLAG: --lock-file="" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474089 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474092 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474095 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474101 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474103 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474110 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474113 2563 flags.go:64] FLAG: --logging-format="text" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474118 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474122 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474125 2563 flags.go:64] FLAG: --manifest-url="" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474128 2563 flags.go:64] FLAG: --manifest-url-header="" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474133 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474137 2563 flags.go:64] FLAG: --max-open-files="1000000" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474141 2563 flags.go:64] FLAG: --max-pods="110" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474144 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474147 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474150 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474153 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474156 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474159 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474162 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 11 21:10:26.482239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474169 2563 flags.go:64] FLAG: --node-status-max-images="50" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474172 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474175 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474178 2563 flags.go:64] FLAG: --pod-cidr="" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474181 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4a2e90c8ef7d783e4d635e24fb09435f5e1f6e954583512c0944f9e57bc77b52" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474187 2563 flags.go:64] FLAG: --pod-manifest-path="" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474190 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474193 2563 flags.go:64] FLAG: --pods-per-core="0" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474196 2563 flags.go:64] FLAG: --port="10250" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474199 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474202 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fc0bcd50963856ff" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474205 2563 flags.go:64] FLAG: --qos-reserved="" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474208 2563 flags.go:64] FLAG: --read-only-port="10255" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474211 2563 flags.go:64] FLAG: --register-node="true" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474214 2563 flags.go:64] FLAG: --register-with-taints="" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474218 2563 flags.go:64] FLAG: --registry-burst="10" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474221 2563 flags.go:64] FLAG: --registry-qps="5" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474226 2563 flags.go:64] FLAG: --reserved-cpus="" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474230 2563 flags.go:64] FLAG: --reserved-memory="" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474233 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474237 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474241 2563 flags.go:64] FLAG: --rotate-certificates="false" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474244 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474247 2563 flags.go:64] FLAG: --runonce="false" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474250 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 11 21:10:26.483359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474253 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474256 2563 flags.go:64] FLAG: --seccomp-default="false" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474259 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474262 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474265 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474268 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474271 2563 flags.go:64] FLAG: --storage-driver-password="root" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474274 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474277 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474281 2563 flags.go:64] FLAG: --storage-driver-user="root" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474283 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474286 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474289 2563 flags.go:64] FLAG: --system-cgroups="" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474292 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474299 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474302 2563 flags.go:64] FLAG: --tls-cert-file="" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474305 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474309 2563 flags.go:64] FLAG: --tls-min-version="" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474312 2563 flags.go:64] FLAG: --tls-private-key-file="" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474315 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474318 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474321 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474324 2563 flags.go:64] FLAG: --v="2" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474328 2563 flags.go:64] FLAG: --version="false" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474333 2563 flags.go:64] FLAG: --vmodule="" Jun 11 21:10:26.484423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474338 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.474342 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474464 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474468 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474472 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474475 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474478 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474481 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474486 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474490 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474493 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474495 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474498 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474501 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474504 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474506 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474509 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474512 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474514 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474517 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 21:10:26.485114 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474520 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474523 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474526 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474529 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474531 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474534 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474536 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474539 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474541 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474544 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474546 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474550 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474554 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474557 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474560 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474562 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474565 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474567 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474570 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474573 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 21:10:26.485616 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474576 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474578 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474581 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474583 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474586 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474588 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474591 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474594 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474596 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474599 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474601 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474604 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474606 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474608 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474611 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474613 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474616 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474618 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474621 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474623 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 21:10:26.486130 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474626 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474628 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474631 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474634 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474638 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474641 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474644 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474648 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474651 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474654 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474657 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474659 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474662 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474664 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474667 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474669 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474672 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474674 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474677 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 21:10:26.486650 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474679 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474682 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474684 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474687 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474689 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474692 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474694 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474697 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474701 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474704 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474707 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474710 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474713 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474716 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474719 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474722 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474725 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474729 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474732 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 21:10:26.487150 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474734 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 21:10:26.487614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474737 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 21:10:26.487614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.474740 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 21:10:26.487614 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.475276 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 21:10:26.487614 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.483253 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 11 21:10:26.487614 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.483271 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 11 21:10:26.487614 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.483297 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 11 21:10:26.487614 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.483306 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 11 21:10:26.487614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484899 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 21:10:26.487614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484913 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 21:10:26.487614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484926 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 21:10:26.487614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484930 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 21:10:26.487614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484933 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 21:10:26.487614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484937 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 21:10:26.487614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484940 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 21:10:26.487614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484943 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484946 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484948 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484951 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484954 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484956 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484959 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484962 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484964 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484967 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484970 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484973 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484976 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484978 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484981 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484984 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484987 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484990 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484992 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484995 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 21:10:26.488041 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.484998 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485000 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485003 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485006 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485009 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485012 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485016 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485034 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485037 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485040 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485042 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485045 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485048 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485050 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485053 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485056 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485058 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485061 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485064 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 21:10:26.488546 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485066 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485069 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485074 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485078 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485087 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485090 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485093 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485097 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485100 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485103 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485106 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485108 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485111 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485114 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485117 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485122 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485125 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485128 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485131 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 21:10:26.489160 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485133 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485136 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485139 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485141 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485144 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485147 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485149 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485153 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485156 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485159 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485162 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485165 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485167 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485170 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485173 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485175 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485178 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485180 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485183 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485186 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 21:10:26.489687 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485189 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 21:10:26.490235 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485191 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 21:10:26.490235 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485194 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 21:10:26.490235 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485197 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 21:10:26.490235 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485199 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 21:10:26.490235 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485202 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 21:10:26.490235 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485204 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 21:10:26.490235 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485207 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 21:10:26.490235 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485210 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 21:10:26.490235 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485212 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 21:10:26.490235 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485215 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 21:10:26.490235 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485218 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 21:10:26.490235 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485220 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 21:10:26.490235 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485223 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 21:10:26.490235 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.485228 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 21:10:26.490235 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485366 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485372 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485375 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485378 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485381 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485383 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485386 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485388 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485391 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485394 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485397 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485399 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485402 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485404 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485407 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485409 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485412 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485416 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485420 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485423 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 21:10:26.490614 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485426 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485429 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485431 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485434 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485437 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485439 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485442 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485446 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485449 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485452 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485455 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485458 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485461 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485463 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485466 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485469 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485471 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485474 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485476 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 21:10:26.491202 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485479 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485481 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485484 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485486 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485489 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485491 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485493 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485496 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485499 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485501 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485504 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485507 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485509 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485512 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485514 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485517 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485520 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485523 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485527 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 21:10:26.491913 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485530 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485532 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485535 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485537 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485539 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485542 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485544 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485547 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485550 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485553 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485555 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485558 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485560 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485563 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485565 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485568 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485570 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485573 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485575 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485577 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 21:10:26.492603 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485580 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485582 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485585 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485587 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485591 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485593 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485596 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485598 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485601 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485603 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485606 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485609 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485611 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485614 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485616 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485619 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485622 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485625 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485627 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485633 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 21:10:26.493239 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:26.485635 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 21:10:26.493863 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.485641 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 21:10:26.493863 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.485756 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jun 11 21:10:26.493863 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.487749 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 11 21:10:26.493863 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.488998 2563 server.go:1019] "Starting client certificate rotation" Jun 11 21:10:26.493863 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.489120 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 21:10:26.493863 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.489157 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 21:10:26.493863 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.491230 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 21:10:26.495056 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.495042 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 21:10:26.502170 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.502154 2563 log.go:25] "Validated CRI v1 runtime API" Jun 11 21:10:26.507067 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.507053 2563 log.go:25] "Validated CRI v1 image API" Jun 11 21:10:26.507904 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.507885 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 11 21:10:26.511839 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.511818 2563 fs.go:135] Filesystem UUIDs: map[02d12467-78af-4c97-8dd2-a498b18332c3:/dev/nvme0n1p3 4e5762ab-70f4-4900-9ac6-ac0adb2bc59a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jun 11 21:10:26.511911 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.511837 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 11 21:10:26.517700 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.517681 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 21:10:26.518319 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.518212 2563 manager.go:216] Machine: {Timestamp:2026-06-11 21:10:26.517126582 +0000 UTC m=+0.260206562 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099781 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2172e8064bb10e40372889ad7d828d SystemUUID:ec2172e8-064b-b10e-4037-2889ad7d828d BootID:4fa8ca42-4930-406d-be60-d465f11a3e45 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d8:75:e1:ce:cf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d8:75:e1:ce:cf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:b2:6b:54:11:b9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 11 21:10:26.518319 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.518315 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 11 21:10:26.518458 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.518414 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.120.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260608-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 11 21:10:26.521235 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.521195 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 11 21:10:26.521381 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.521243 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-167.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 11 21:10:26.522285 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.522275 2563 topology_manager.go:138] "Creating topology manager with none policy" Jun 11 21:10:26.522325 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.522295 2563 container_manager_linux.go:309] "Creating device plugin manager" Jun 11 21:10:26.522325 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.522303 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 21:10:26.522830 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.522821 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 21:10:26.523993 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.523983 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 11 21:10:26.526579 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.526567 2563 state_mem.go:36] "Initialized new in-memory state store" Jun 11 21:10:26.527146 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.527073 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 11 21:10:26.527275 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.527201 2563 kubelet.go:486] "Attempting to sync node with API server" Jun 11 21:10:26.527275 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.527240 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 11 21:10:26.528227 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.527384 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 11 21:10:26.528227 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.527402 2563 kubelet.go:398] "Adding apiserver pod source" Jun 11 21:10:26.528227 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.527446 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 11 21:10:26.528227 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.527992 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 21:10:26.528227 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.528014 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 21:10:26.530419 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.530398 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 11 21:10:26.531881 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.531868 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 11 21:10:26.531931 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.531898 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 11 21:10:26.532099 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.532087 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 11 21:10:26.532160 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.532107 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 11 21:10:26.532160 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.532117 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 11 21:10:26.532160 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.532126 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 11 21:10:26.532160 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.532137 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 11 21:10:26.532160 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.532145 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 11 21:10:26.532551 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.532540 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 11 21:10:26.532551 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.532551 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 11 21:10:26.532634 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.532559 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 11 21:10:26.532634 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.532564 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 11 21:10:26.532634 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.532572 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 11 21:10:26.532634 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.532582 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 11 21:10:26.533113 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.533103 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 11 21:10:26.533113 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.533113 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 11 21:10:26.536360 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.536347 2563 server.go:1268] "Started kubelet" Jun 11 21:10:26.536445 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.536411 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 11 21:10:26.536494 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.536437 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 11 21:10:26.536543 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.536517 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 11 21:10:26.537269 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.537248 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xldtp" Jun 11 21:10:26.537353 ip-10-0-140-167 systemd[1]: Started Kubernetes Kubelet. Jun 11 21:10:26.537970 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.537954 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 11 21:10:26.538277 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.538260 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-167.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 11 21:10:26.538347 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.538269 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 11 21:10:26.538456 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.538434 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-167.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 11 21:10:26.539002 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.538984 2563 server.go:310] "Adding debug handlers to kubelet server" Jun 11 21:10:26.542963 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.542945 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xldtp" Jun 11 21:10:26.543406 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.543392 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 11 21:10:26.543489 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.543409 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 11 21:10:26.543489 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.543469 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:26.544118 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.544100 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 11 21:10:26.544118 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.544103 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 11 21:10:26.544245 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.544127 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 11 21:10:26.544288 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.544249 2563 reconstruct.go:97] "Volume reconstruction finished" Jun 11 21:10:26.544288 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.544260 2563 reconciler.go:29] "Reconciler: start to sync state" Jun 11 21:10:26.544549 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.544531 2563 factory.go:55] Registering systemd factory Jun 11 21:10:26.544628 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.544554 2563 factory.go:223] Registration of the systemd container factory successfully Jun 11 21:10:26.544762 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.544743 2563 factory.go:153] Registering CRI-O factory Jun 11 21:10:26.544852 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.544765 2563 factory.go:223] Registration of the crio container factory successfully Jun 11 21:10:26.544852 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.544814 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 11 21:10:26.544852 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.544849 2563 factory.go:103] Registering Raw factory Jun 11 21:10:26.544989 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.544866 2563 manager.go:1196] Started watching for new ooms in manager Jun 11 21:10:26.545350 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.545332 2563 manager.go:349] Starting recovery of all containers Jun 11 21:10:26.545497 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.544752 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-167.ec2.internal.18b823153047afc0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-167.ec2.internal,UID:ip-10-0-140-167.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-167.ec2.internal,},FirstTimestamp:2026-06-11 21:10:26.53632096 +0000 UTC m=+0.279400939,LastTimestamp:2026-06-11 21:10:26.53632096 +0000 UTC m=+0.279400939,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-167.ec2.internal,}" Jun 11 21:10:26.545618 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.545593 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 11 21:10:26.549122 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.549098 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 21:10:26.551825 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.551801 2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-167.ec2.internal\" not found" node="ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.557850 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.557828 2563 manager.go:354] Recovery completed Jun 11 21:10:26.562992 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.562977 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 21:10:26.565611 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.565591 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientMemory" Jun 11 21:10:26.565696 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.565624 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 21:10:26.565696 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.565637 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientPID" Jun 11 21:10:26.566137 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.566123 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 11 21:10:26.566137 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.566133 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 21:10:26.566137 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.566137 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 11 21:10:26.566304 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.566157 2563 state_mem.go:36] "Initialized new in-memory state store" Jun 11 21:10:26.566881 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.566868 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientMemory" Jun 11 21:10:26.566939 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.566895 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 21:10:26.566939 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.566907 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientPID" Jun 11 21:10:26.567220 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.567207 2563 policy_none.go:49] "None policy: Start" Jun 11 21:10:26.567285 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.567227 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 11 21:10:26.567285 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.567243 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 11 21:10:26.568219 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.568146 2563 policy_none.go:47] "Start" Jun 11 21:10:26.608819 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.608794 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 11 21:10:26.609990 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.609779 2563 manager.go:337] "Starting Device Plugin manager" Jun 11 21:10:26.609990 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.609810 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 11 21:10:26.609990 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.609821 2563 server.go:87] "Starting device plugin registration server" Jun 11 21:10:26.610167 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.610125 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 11 21:10:26.610167 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.610146 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 11 21:10:26.610516 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.610341 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 11 21:10:26.610516 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.610446 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 11 21:10:26.610516 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.610456 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 11 21:10:26.610693 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.610546 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 11 21:10:26.610693 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.610604 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 11 21:10:26.610693 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.610655 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jun 11 21:10:26.611174 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.611050 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 11 21:10:26.611174 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.611001 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 11 21:10:26.611174 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.611105 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:26.612508 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.612482 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 21:10:26.711185 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.711155 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 21:10:26.711185 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.711154 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal"] Jun 11 21:10:26.711383 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.711255 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 21:10:26.712255 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.712237 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientMemory" Jun 11 21:10:26.712358 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.712265 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 21:10:26.712358 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.712278 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientPID" Jun 11 21:10:26.712358 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.712243 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientMemory" Jun 11 21:10:26.712358 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.712335 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 21:10:26.712358 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.712358 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientPID" Jun 11 21:10:26.712503 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.712382 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.713558 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.713546 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 21:10:26.713657 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.713634 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.713762 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.713680 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 21:10:26.715109 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.715053 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientMemory" Jun 11 21:10:26.715109 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.715086 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 21:10:26.715109 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.715103 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientPID" Jun 11 21:10:26.715236 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.715060 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientMemory" Jun 11 21:10:26.715236 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.715158 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 21:10:26.715236 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.715174 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientPID" Jun 11 21:10:26.716261 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.716246 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.716336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.716268 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 21:10:26.717984 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.717969 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.718067 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.717987 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-167.ec2.internal\": node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:26.719873 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.719857 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientMemory" Jun 11 21:10:26.719963 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.719878 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 21:10:26.719963 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.719898 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeHasSufficientPID" Jun 11 21:10:26.730607 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.730585 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:26.741737 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.741710 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-167.ec2.internal\" not found" node="ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.745153 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.745136 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dc1c0b2bc241594d3e7f1c749bf01355-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal\" (UID: \"dc1c0b2bc241594d3e7f1c749bf01355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.745246 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.745160 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc1c0b2bc241594d3e7f1c749bf01355-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal\" (UID: \"dc1c0b2bc241594d3e7f1c749bf01355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.745246 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.745175 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5b0fb3102504571ccdfcf8b539950afa-config\") pod \"kube-apiserver-proxy-ip-10-0-140-167.ec2.internal\" (UID: \"5b0fb3102504571ccdfcf8b539950afa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.746195 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.746180 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-167.ec2.internal\" not found" node="ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.831448 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.831418 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:26.845297 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.845270 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dc1c0b2bc241594d3e7f1c749bf01355-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal\" (UID: \"dc1c0b2bc241594d3e7f1c749bf01355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.845406 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.845300 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc1c0b2bc241594d3e7f1c749bf01355-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal\" (UID: \"dc1c0b2bc241594d3e7f1c749bf01355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.845406 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.845323 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5b0fb3102504571ccdfcf8b539950afa-config\") pod \"kube-apiserver-proxy-ip-10-0-140-167.ec2.internal\" (UID: \"5b0fb3102504571ccdfcf8b539950afa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.845406 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.845349 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dc1c0b2bc241594d3e7f1c749bf01355-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal\" (UID: \"dc1c0b2bc241594d3e7f1c749bf01355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.845406 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.845350 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5b0fb3102504571ccdfcf8b539950afa-config\") pod \"kube-apiserver-proxy-ip-10-0-140-167.ec2.internal\" (UID: \"5b0fb3102504571ccdfcf8b539950afa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.845406 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:26.845371 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc1c0b2bc241594d3e7f1c749bf01355-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal\" (UID: \"dc1c0b2bc241594d3e7f1c749bf01355\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" Jun 11 21:10:26.932292 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:26.932256 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:27.033014 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:27.032933 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:27.044097 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:27.044073 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" Jun 11 21:10:27.049211 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:27.049075 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal" Jun 11 21:10:27.133738 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:27.133703 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:27.234218 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:27.234191 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:27.334771 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:27.334680 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:27.411313 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:27.411286 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 21:10:27.435054 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:27.435014 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:27.488520 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:27.488490 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 11 21:10:27.489146 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:27.488678 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 21:10:27.489146 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:27.488680 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 21:10:27.489146 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:27.488711 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 21:10:27.535627 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:27.535581 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:27.546683 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:27.546650 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-10 21:05:26 +0000 UTC" deadline="2028-02-22 20:57:19.224090138 +0000 UTC" Jun 11 21:10:27.546683 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:27.546679 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14903h46m51.677414462s" Jun 11 21:10:27.552008 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:27.551982 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b0fb3102504571ccdfcf8b539950afa.slice/crio-cf56a30268d7a3c565438429ad65bc985ea86482449fb4b8394b1cb154e6438d WatchSource:0}: Error finding container cf56a30268d7a3c565438429ad65bc985ea86482449fb4b8394b1cb154e6438d: Status 404 returned error can't find the container with id cf56a30268d7a3c565438429ad65bc985ea86482449fb4b8394b1cb154e6438d Jun 11 21:10:27.552235 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:27.552220 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc1c0b2bc241594d3e7f1c749bf01355.slice/crio-2faf5afadcb7ecf8bb2c82c8e02ba29b89b1a59cb0a6d03aeddb0c5369fc496f WatchSource:0}: Error finding container 2faf5afadcb7ecf8bb2c82c8e02ba29b89b1a59cb0a6d03aeddb0c5369fc496f: Status 404 returned error can't find the container with id 2faf5afadcb7ecf8bb2c82c8e02ba29b89b1a59cb0a6d03aeddb0c5369fc496f Jun 11 21:10:27.556676 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:27.556656 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 21:10:27.614471 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:27.614360 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" event={"ID":"dc1c0b2bc241594d3e7f1c749bf01355","Type":"ContainerStarted","Data":"2faf5afadcb7ecf8bb2c82c8e02ba29b89b1a59cb0a6d03aeddb0c5369fc496f"} Jun 11 21:10:27.615394 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:27.615372 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal" event={"ID":"5b0fb3102504571ccdfcf8b539950afa","Type":"ContainerStarted","Data":"cf56a30268d7a3c565438429ad65bc985ea86482449fb4b8394b1cb154e6438d"} Jun 11 21:10:27.636557 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:27.636527 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:27.737133 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:27.737099 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:27.837537 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:27.837511 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:27.938075 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:27.938041 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:28.038828 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:28.038796 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-167.ec2.internal\" not found" Jun 11 21:10:28.048996 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.048959 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 21:10:28.144806 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.144777 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" Jun 11 21:10:28.155257 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.155110 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 21:10:28.156080 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.156061 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal" Jun 11 21:10:28.163438 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.163420 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 21:10:28.529409 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.529332 2563 apiserver.go:52] "Watching apiserver" Jun 11 21:10:28.537818 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.537795 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 11 21:10:28.538257 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.538230 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-bqqv5","kube-system/konnectivity-agent-l4pjg","kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67","openshift-cluster-node-tuning-operator/tuned-wb646","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal","openshift-multus/multus-additional-cni-plugins-mkwd5","openshift-multus/multus-vtjdn","openshift-dns/node-resolver-9wcfm","openshift-image-registry/node-ca-vb7pc","openshift-multus/network-metrics-daemon-hz8r8","openshift-network-diagnostics/network-check-target-g9ff4","openshift-network-operator/iptables-alerter-b6xn8","openshift-ovn-kubernetes/ovnkube-node-qhxkp"] Jun 11 21:10:28.539733 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.539711 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.540868 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.540841 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b6xn8" Jun 11 21:10:28.542158 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.542139 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 11 21:10:28.542298 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.542275 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 11 21:10:28.542402 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.542141 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.542402 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.542377 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 11 21:10:28.542402 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.542275 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 11 21:10:28.542546 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.542532 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 11 21:10:28.542621 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.542601 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ct8dh\"" Jun 11 21:10:28.543198 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.543176 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 11 21:10:28.543277 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.543217 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 11 21:10:28.543277 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.543242 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fpgfr\"" Jun 11 21:10:28.543398 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.543330 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.544351 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.544330 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 11 21:10:28.544581 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.544564 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.544675 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.544569 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 11 21:10:28.545487 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.545284 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 11 21:10:28.545487 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.545471 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 11 21:10:28.545676 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.545548 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 11 21:10:28.545676 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.545663 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 11 21:10:28.545768 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.545692 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 11 21:10:28.546431 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.546409 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6htvx\"" Jun 11 21:10:28.547298 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.546733 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 11 21:10:28.547298 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.547091 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.547452 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.547434 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-f766s\"" Jun 11 21:10:28.547536 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.547510 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 11 21:10:28.547785 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.547766 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 11 21:10:28.547900 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.547770 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 11 21:10:28.548866 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.548844 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rdn6z\"" Jun 11 21:10:28.549271 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.549239 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 11 21:10:28.549659 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.549639 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 11 21:10:28.549867 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.549846 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wjxvz\"" Jun 11 21:10:28.549943 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.549890 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 11 21:10:28.550678 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.550446 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9wcfm" Jun 11 21:10:28.552290 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.552263 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:28.552372 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:28.552354 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:28.552482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.552464 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lsxc6\"" Jun 11 21:10:28.552786 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.552747 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 11 21:10:28.552910 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.552890 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 11 21:10:28.553725 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.553702 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-run-ovn\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.553830 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.553738 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-log-socket\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.553830 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.553758 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l4pjg" Jun 11 21:10:28.553830 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.553764 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-cni-netd\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.553830 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.553789 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-ovnkube-script-lib\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.553830 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.553811 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-device-dir\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.553830 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.553828 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38607a8f-f08d-4fce-a21a-c885cc677d51-cnibin\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.554155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.553843 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckt7w\" (UniqueName: \"kubernetes.io/projected/38607a8f-f08d-4fce-a21a-c885cc677d51-kube-api-access-ckt7w\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.554155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.553857 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/33f3fb6a-f938-457d-9905-b4e1d7b115a0-iptables-alerter-script\") pod \"iptables-alerter-b6xn8\" (UID: \"33f3fb6a-f938-457d-9905-b4e1d7b115a0\") " pod="openshift-network-operator/iptables-alerter-b6xn8" Jun 11 21:10:28.554155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.553890 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.554155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.553893 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 21:10:28.554155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.553905 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-sysctl-d\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.554155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.553930 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-systemd\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.554155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.553953 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-tuned\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.554155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.553977 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33f3fb6a-f938-457d-9905-b4e1d7b115a0-host-slash\") pod \"iptables-alerter-b6xn8\" (UID: \"33f3fb6a-f938-457d-9905-b4e1d7b115a0\") " pod="openshift-network-operator/iptables-alerter-b6xn8" Jun 11 21:10:28.554155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554001 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-systemd-units\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.554155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554079 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-slash\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.554155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554104 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-run-netns\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.554155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554127 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-var-lib-openvswitch\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.554155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554156 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-node-log\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554181 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-cni-bin\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554214 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-env-overrides\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554260 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbmqh\" (UniqueName: \"kubernetes.io/projected/933eb3d8-8042-429b-b122-83d4705c128c-kube-api-access-jbmqh\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554290 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-modprobe-d\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554306 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-sysconfig\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554322 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-lib-modules\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554335 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-tmp\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554350 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/38607a8f-f08d-4fce-a21a-c885cc677d51-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554374 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-kubelet\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554397 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-ovnkube-config\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554434 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjjsm\" (UniqueName: \"kubernetes.io/projected/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-kube-api-access-vjjsm\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554456 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-socket-dir\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554478 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-sys\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554494 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-host\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554527 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-run-openvswitch\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.554743 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554573 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-run-ovn-kubernetes\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554601 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-ovn-node-metrics-cert\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554634 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-sys-fs\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554657 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-kubernetes\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554679 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-var-lib-kubelet\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554704 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2mbs\" (UniqueName: \"kubernetes.io/projected/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-kube-api-access-x2mbs\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554726 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38607a8f-f08d-4fce-a21a-c885cc677d51-system-cni-dir\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554750 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554773 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-registration-dir\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554801 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/38607a8f-f08d-4fce-a21a-c885cc677d51-os-release\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554854 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38607a8f-f08d-4fce-a21a-c885cc677d51-cni-binary-copy\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554880 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghl8s\" (UniqueName: \"kubernetes.io/projected/33f3fb6a-f938-457d-9905-b4e1d7b115a0-kube-api-access-ghl8s\") pod \"iptables-alerter-b6xn8\" (UID: \"33f3fb6a-f938-457d-9905-b4e1d7b115a0\") " pod="openshift-network-operator/iptables-alerter-b6xn8" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554905 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-run-systemd\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554936 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-etc-openvswitch\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554959 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.554982 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-sysctl-conf\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.555482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.555004 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-run\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.556252 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.555041 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38607a8f-f08d-4fce-a21a-c885cc677d51-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.556252 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.555066 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/38607a8f-f08d-4fce-a21a-c885cc677d51-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.556252 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.555143 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vb7pc" Jun 11 21:10:28.556252 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.555991 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jhvxt\"" Jun 11 21:10:28.556252 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.556036 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 11 21:10:28.556252 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.556062 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 11 21:10:28.556640 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.556397 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:28.556640 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:28.556466 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:28.556908 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.556888 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 11 21:10:28.557314 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.557296 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 11 21:10:28.557385 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.557367 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xghsj\"" Jun 11 21:10:28.557429 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.557367 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 11 21:10:28.557732 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.557712 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:28.557818 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:28.557770 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:28.574145 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.574126 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-24tml" Jun 11 21:10:28.580969 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.580948 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-24tml" Jun 11 21:10:28.645199 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.645173 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 11 21:10:28.655299 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655266 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-etc-openvswitch\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.655299 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655298 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.655509 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655325 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-sysctl-conf\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.655509 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655401 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-etc-openvswitch\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.655509 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655422 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.655509 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655451 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-run\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.655509 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655474 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-sysctl-conf\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.655509 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655483 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38607a8f-f08d-4fce-a21a-c885cc677d51-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.655809 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655510 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/38607a8f-f08d-4fce-a21a-c885cc677d51-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.655809 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655541 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-etc-kubernetes\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.655809 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655541 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-run\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.655809 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655564 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a76fe3c1-133a-4c0d-b36f-4f309a013579-tmp-dir\") pod \"node-resolver-9wcfm\" (UID: \"a76fe3c1-133a-4c0d-b36f-4f309a013579\") " pod="openshift-dns/node-resolver-9wcfm" Jun 11 21:10:28.655809 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655607 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-ovnkube-script-lib\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.655809 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655643 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33f3fb6a-f938-457d-9905-b4e1d7b115a0-host-slash\") pod \"iptables-alerter-b6xn8\" (UID: \"33f3fb6a-f938-457d-9905-b4e1d7b115a0\") " pod="openshift-network-operator/iptables-alerter-b6xn8" Jun 11 21:10:28.655809 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655701 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38607a8f-f08d-4fce-a21a-c885cc677d51-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.655809 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655730 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/23d74d65-50c4-45b2-bfb0-33bd224279c8-serviceca\") pod \"node-ca-vb7pc\" (UID: \"23d74d65-50c4-45b2-bfb0-33bd224279c8\") " pod="openshift-image-registry/node-ca-vb7pc" Jun 11 21:10:28.655809 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655732 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33f3fb6a-f938-457d-9905-b4e1d7b115a0-host-slash\") pod \"iptables-alerter-b6xn8\" (UID: \"33f3fb6a-f938-457d-9905-b4e1d7b115a0\") " pod="openshift-network-operator/iptables-alerter-b6xn8" Jun 11 21:10:28.655809 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655772 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a76fe3c1-133a-4c0d-b36f-4f309a013579-hosts-file\") pod \"node-resolver-9wcfm\" (UID: \"a76fe3c1-133a-4c0d-b36f-4f309a013579\") " pod="openshift-dns/node-resolver-9wcfm" Jun 11 21:10:28.655809 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655803 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-systemd\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655826 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/58f726ff-e37a-497f-8a7c-9216791bdd00-kubelet-config\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655852 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86d5c\" (UniqueName: \"kubernetes.io/projected/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-kube-api-access-86d5c\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655870 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-systemd\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655876 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-slash\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655916 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-node-log\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655926 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-slash\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655967 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-node-log\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.655977 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-env-overrides\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656007 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-tmp\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656111 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/38607a8f-f08d-4fce-a21a-c885cc677d51-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656172 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-system-cni-dir\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656202 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-cni-binary-copy\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656212 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-ovnkube-script-lib\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656227 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-var-lib-cni-bin\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656253 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-var-lib-cni-multus\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656297 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-kubelet\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.656336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656323 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-env-overrides\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.657059 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656539 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-socket-dir\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.657059 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656786 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-host\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.657059 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656828 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2mbs\" (UniqueName: \"kubernetes.io/projected/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-kube-api-access-x2mbs\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.657059 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656834 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-kubelet\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.657059 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656854 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-socket-dir\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.657059 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656862 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38607a8f-f08d-4fce-a21a-c885cc677d51-system-cni-dir\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.657059 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656912 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/58f726ff-e37a-497f-8a7c-9216791bdd00-dbus\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:28.657059 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656920 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 11 21:10:28.657059 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656974 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-host\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.657059 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.656991 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38607a8f-f08d-4fce-a21a-c885cc677d51-system-cni-dir\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.657059 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657048 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-run-openvswitch\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657090 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-run-openvswitch\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657114 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-run-ovn-kubernetes\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657153 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-ovn-node-metrics-cert\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657177 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-sys-fs\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657195 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-run-ovn-kubernetes\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657207 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-kubernetes\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657249 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-var-lib-kubelet\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657275 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-sys-fs\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657281 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/38607a8f-f08d-4fce-a21a-c885cc677d51-os-release\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657347 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-var-lib-kubelet\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657405 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-kubernetes\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657461 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/38607a8f-f08d-4fce-a21a-c885cc677d51-os-release\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657497 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-cnibin\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657532 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657564 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-var-lib-kubelet\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.657598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657597 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-hostroot\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657633 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckt7w\" (UniqueName: \"kubernetes.io/projected/38607a8f-f08d-4fce-a21a-c885cc677d51-kube-api-access-ckt7w\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657658 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-run-systemd\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657671 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657686 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-multus-conf-dir\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657717 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aea1a453-0093-47c5-87bb-67c9fa59021c-agent-certs\") pod \"konnectivity-agent-l4pjg\" (UID: \"aea1a453-0093-47c5-87bb-67c9fa59021c\") " pod="kube-system/konnectivity-agent-l4pjg" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657744 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-run-ovn\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657775 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-log-socket\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657798 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-cni-netd\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657828 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-device-dir\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657860 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38607a8f-f08d-4fce-a21a-c885cc677d51-cnibin\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657889 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/33f3fb6a-f938-457d-9905-b4e1d7b115a0-iptables-alerter-script\") pod \"iptables-alerter-b6xn8\" (UID: \"33f3fb6a-f938-457d-9905-b4e1d7b115a0\") " pod="openshift-network-operator/iptables-alerter-b6xn8" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657914 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.657967 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-multus-cni-dir\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658001 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658049 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-sysctl-d\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658079 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-tuned\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.658278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658108 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m45vc\" (UniqueName: \"kubernetes.io/projected/db4e8904-7461-4943-909f-6e9fe29dea61-kube-api-access-m45vc\") pod \"network-metrics-daemon-hz8r8\" (UID: \"db4e8904-7461-4943-909f-6e9fe29dea61\") " pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658141 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t584b\" (UniqueName: \"kubernetes.io/projected/23d74d65-50c4-45b2-bfb0-33bd224279c8-kube-api-access-t584b\") pod \"node-ca-vb7pc\" (UID: \"23d74d65-50c4-45b2-bfb0-33bd224279c8\") " pod="openshift-image-registry/node-ca-vb7pc" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658173 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zndx6\" (UniqueName: \"kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6\") pod \"network-check-target-g9ff4\" (UID: \"c769bcd4-a84b-488c-a780-2d5b36778839\") " pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658205 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-systemd-units\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658237 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-run-netns\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658263 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-var-lib-openvswitch\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658296 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-cni-bin\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658328 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbmqh\" (UniqueName: \"kubernetes.io/projected/933eb3d8-8042-429b-b122-83d4705c128c-kube-api-access-jbmqh\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658362 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-modprobe-d\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658388 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-multus-socket-dir-parent\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658419 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-run-netns\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658450 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-sysconfig\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658481 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-lib-modules\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658517 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/38607a8f-f08d-4fce-a21a-c885cc677d51-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658547 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs\") pod \"network-metrics-daemon-hz8r8\" (UID: \"db4e8904-7461-4943-909f-6e9fe29dea61\") " pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658578 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-run-k8s-cni-cncf-io\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658604 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aea1a453-0093-47c5-87bb-67c9fa59021c-konnectivity-ca\") pod \"konnectivity-agent-l4pjg\" (UID: \"aea1a453-0093-47c5-87bb-67c9fa59021c\") " pod="kube-system/konnectivity-agent-l4pjg" Jun 11 21:10:28.658991 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658635 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-ovnkube-config\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658666 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjjsm\" (UniqueName: \"kubernetes.io/projected/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-kube-api-access-vjjsm\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658696 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-sys\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658717 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-sysctl-d\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658735 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-multus-daemon-config\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658763 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-run-multus-certs\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658794 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23d74d65-50c4-45b2-bfb0-33bd224279c8-host\") pod \"node-ca-vb7pc\" (UID: \"23d74d65-50c4-45b2-bfb0-33bd224279c8\") " pod="openshift-image-registry/node-ca-vb7pc" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658825 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38607a8f-f08d-4fce-a21a-c885cc677d51-cni-binary-copy\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658857 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghl8s\" (UniqueName: \"kubernetes.io/projected/33f3fb6a-f938-457d-9905-b4e1d7b115a0-kube-api-access-ghl8s\") pod \"iptables-alerter-b6xn8\" (UID: \"33f3fb6a-f938-457d-9905-b4e1d7b115a0\") " pod="openshift-network-operator/iptables-alerter-b6xn8" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658885 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-os-release\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658917 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-registration-dir\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.658951 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7xr2\" (UniqueName: \"kubernetes.io/projected/a76fe3c1-133a-4c0d-b36f-4f309a013579-kube-api-access-h7xr2\") pod \"node-resolver-9wcfm\" (UID: \"a76fe3c1-133a-4c0d-b36f-4f309a013579\") " pod="openshift-dns/node-resolver-9wcfm" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.659060 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-run-systemd\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.659154 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-run-ovn\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.659200 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-log-socket\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.659247 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-cni-netd\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.659310 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-device-dir\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.659789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.659359 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38607a8f-f08d-4fce-a21a-c885cc677d51-cnibin\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.660574 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.659835 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/38607a8f-f08d-4fce-a21a-c885cc677d51-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.660574 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.660048 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38607a8f-f08d-4fce-a21a-c885cc677d51-cni-binary-copy\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.660574 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.660110 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-systemd-units\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.660574 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.660123 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.660574 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.660165 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-run-netns\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.660574 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.660209 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-var-lib-openvswitch\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.660574 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.660250 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-host-cni-bin\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.660574 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.660340 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-modprobe-d\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.660574 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.660389 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-sysconfig\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.660574 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.660468 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-lib-modules\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.660574 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.660519 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/933eb3d8-8042-429b-b122-83d4705c128c-registration-dir\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.660574 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.660573 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-ovnkube-config\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.661412 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.660613 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/33f3fb6a-f938-457d-9905-b4e1d7b115a0-iptables-alerter-script\") pod \"iptables-alerter-b6xn8\" (UID: \"33f3fb6a-f938-457d-9905-b4e1d7b115a0\") " pod="openshift-network-operator/iptables-alerter-b6xn8" Jun 11 21:10:28.661412 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.660672 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-sys\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.661509 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.661450 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-ovn-node-metrics-cert\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.662232 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.661829 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-etc-tuned\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.663277 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.663225 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-tmp\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.665183 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.665139 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2mbs\" (UniqueName: \"kubernetes.io/projected/ca2f91ff-0e01-49e9-9bf1-3552f79b1f15-kube-api-access-x2mbs\") pod \"tuned-wb646\" (UID: \"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15\") " pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.665880 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.665813 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckt7w\" (UniqueName: \"kubernetes.io/projected/38607a8f-f08d-4fce-a21a-c885cc677d51-kube-api-access-ckt7w\") pod \"multus-additional-cni-plugins-mkwd5\" (UID: \"38607a8f-f08d-4fce-a21a-c885cc677d51\") " pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.667848 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.667798 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjjsm\" (UniqueName: \"kubernetes.io/projected/49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f-kube-api-access-vjjsm\") pod \"ovnkube-node-qhxkp\" (UID: \"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.668584 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.668543 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghl8s\" (UniqueName: \"kubernetes.io/projected/33f3fb6a-f938-457d-9905-b4e1d7b115a0-kube-api-access-ghl8s\") pod \"iptables-alerter-b6xn8\" (UID: \"33f3fb6a-f938-457d-9905-b4e1d7b115a0\") " pod="openshift-network-operator/iptables-alerter-b6xn8" Jun 11 21:10:28.668668 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.668599 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbmqh\" (UniqueName: \"kubernetes.io/projected/933eb3d8-8042-429b-b122-83d4705c128c-kube-api-access-jbmqh\") pod \"aws-ebs-csi-driver-node-fdn67\" (UID: \"933eb3d8-8042-429b-b122-83d4705c128c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.759703 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.759666 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/58f726ff-e37a-497f-8a7c-9216791bdd00-dbus\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:28.759872 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.759720 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-cnibin\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.759872 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.759745 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-var-lib-kubelet\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.759872 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.759768 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-hostroot\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.759872 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.759792 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-multus-conf-dir\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.759872 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.759803 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/58f726ff-e37a-497f-8a7c-9216791bdd00-dbus\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:28.759872 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.759814 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aea1a453-0093-47c5-87bb-67c9fa59021c-agent-certs\") pod \"konnectivity-agent-l4pjg\" (UID: \"aea1a453-0093-47c5-87bb-67c9fa59021c\") " pod="kube-system/konnectivity-agent-l4pjg" Jun 11 21:10:28.759872 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.759854 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-var-lib-kubelet\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.759889 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.759931 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-multus-cni-dir\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.759893 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-cnibin\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.759962 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m45vc\" (UniqueName: \"kubernetes.io/projected/db4e8904-7461-4943-909f-6e9fe29dea61-kube-api-access-m45vc\") pod \"network-metrics-daemon-hz8r8\" (UID: \"db4e8904-7461-4943-909f-6e9fe29dea61\") " pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.759961 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-hostroot\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.759989 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t584b\" (UniqueName: \"kubernetes.io/projected/23d74d65-50c4-45b2-bfb0-33bd224279c8-kube-api-access-t584b\") pod \"node-ca-vb7pc\" (UID: \"23d74d65-50c4-45b2-bfb0-33bd224279c8\") " pod="openshift-image-registry/node-ca-vb7pc" Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760016 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zndx6\" (UniqueName: \"kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6\") pod \"network-check-target-g9ff4\" (UID: \"c769bcd4-a84b-488c-a780-2d5b36778839\") " pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760038 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-multus-cni-dir\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:28.760054 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760065 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-multus-socket-dir-parent\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:28.760124 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret podName:58f726ff-e37a-497f-8a7c-9216791bdd00 nodeName:}" failed. No retries permitted until 2026-06-11 21:10:29.2600953 +0000 UTC m=+3.003175285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret") pod "global-pull-secret-syncer-bqqv5" (UID: "58f726ff-e37a-497f-8a7c-9216791bdd00") : object "kube-system"/"original-pull-secret" not registered Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760132 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-multus-socket-dir-parent\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760142 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-multus-conf-dir\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760208 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-run-netns\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760241 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs\") pod \"network-metrics-daemon-hz8r8\" (UID: \"db4e8904-7461-4943-909f-6e9fe29dea61\") " pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:28.760248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760245 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-run-netns\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760267 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-run-k8s-cni-cncf-io\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760291 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aea1a453-0093-47c5-87bb-67c9fa59021c-konnectivity-ca\") pod \"konnectivity-agent-l4pjg\" (UID: \"aea1a453-0093-47c5-87bb-67c9fa59021c\") " pod="kube-system/konnectivity-agent-l4pjg" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:28.760315 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760320 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-multus-daemon-config\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760344 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-run-multus-certs\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:28.760353 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs podName:db4e8904-7461-4943-909f-6e9fe29dea61 nodeName:}" failed. No retries permitted until 2026-06-11 21:10:29.260338175 +0000 UTC m=+3.003418155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs") pod "network-metrics-daemon-hz8r8" (UID: "db4e8904-7461-4943-909f-6e9fe29dea61") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760373 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-run-k8s-cni-cncf-io\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760392 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-run-multus-certs\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760410 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23d74d65-50c4-45b2-bfb0-33bd224279c8-host\") pod \"node-ca-vb7pc\" (UID: \"23d74d65-50c4-45b2-bfb0-33bd224279c8\") " pod="openshift-image-registry/node-ca-vb7pc" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760439 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-os-release\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760463 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7xr2\" (UniqueName: \"kubernetes.io/projected/a76fe3c1-133a-4c0d-b36f-4f309a013579-kube-api-access-h7xr2\") pod \"node-resolver-9wcfm\" (UID: \"a76fe3c1-133a-4c0d-b36f-4f309a013579\") " pod="openshift-dns/node-resolver-9wcfm" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760493 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-etc-kubernetes\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760507 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23d74d65-50c4-45b2-bfb0-33bd224279c8-host\") pod \"node-ca-vb7pc\" (UID: \"23d74d65-50c4-45b2-bfb0-33bd224279c8\") " pod="openshift-image-registry/node-ca-vb7pc" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760519 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a76fe3c1-133a-4c0d-b36f-4f309a013579-tmp-dir\") pod \"node-resolver-9wcfm\" (UID: \"a76fe3c1-133a-4c0d-b36f-4f309a013579\") " pod="openshift-dns/node-resolver-9wcfm" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760527 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-os-release\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760561 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/23d74d65-50c4-45b2-bfb0-33bd224279c8-serviceca\") pod \"node-ca-vb7pc\" (UID: \"23d74d65-50c4-45b2-bfb0-33bd224279c8\") " pod="openshift-image-registry/node-ca-vb7pc" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760587 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a76fe3c1-133a-4c0d-b36f-4f309a013579-hosts-file\") pod \"node-resolver-9wcfm\" (UID: \"a76fe3c1-133a-4c0d-b36f-4f309a013579\") " pod="openshift-dns/node-resolver-9wcfm" Jun 11 21:10:28.760971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760615 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/58f726ff-e37a-497f-8a7c-9216791bdd00-kubelet-config\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760659 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86d5c\" (UniqueName: \"kubernetes.io/projected/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-kube-api-access-86d5c\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760688 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-system-cni-dir\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760711 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-cni-binary-copy\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760737 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-var-lib-cni-bin\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760752 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/58f726ff-e37a-497f-8a7c-9216791bdd00-kubelet-config\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760795 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-var-lib-cni-multus\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760817 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a76fe3c1-133a-4c0d-b36f-4f309a013579-tmp-dir\") pod \"node-resolver-9wcfm\" (UID: \"a76fe3c1-133a-4c0d-b36f-4f309a013579\") " pod="openshift-dns/node-resolver-9wcfm" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760862 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-etc-kubernetes\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760757 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-var-lib-cni-multus\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760956 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-multus-daemon-config\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760914 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a76fe3c1-133a-4c0d-b36f-4f309a013579-hosts-file\") pod \"node-resolver-9wcfm\" (UID: \"a76fe3c1-133a-4c0d-b36f-4f309a013579\") " pod="openshift-dns/node-resolver-9wcfm" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760978 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aea1a453-0093-47c5-87bb-67c9fa59021c-konnectivity-ca\") pod \"konnectivity-agent-l4pjg\" (UID: \"aea1a453-0093-47c5-87bb-67c9fa59021c\") " pod="kube-system/konnectivity-agent-l4pjg" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.760986 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-host-var-lib-cni-bin\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.761015 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-system-cni-dir\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.761186 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/23d74d65-50c4-45b2-bfb0-33bd224279c8-serviceca\") pod \"node-ca-vb7pc\" (UID: \"23d74d65-50c4-45b2-bfb0-33bd224279c8\") " pod="openshift-image-registry/node-ca-vb7pc" Jun 11 21:10:28.761747 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.761371 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-cni-binary-copy\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.762814 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.762792 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aea1a453-0093-47c5-87bb-67c9fa59021c-agent-certs\") pod \"konnectivity-agent-l4pjg\" (UID: \"aea1a453-0093-47c5-87bb-67c9fa59021c\") " pod="kube-system/konnectivity-agent-l4pjg" Jun 11 21:10:28.766311 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:28.766281 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 21:10:28.766311 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:28.766309 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 21:10:28.766467 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:28.766324 2563 projected.go:196] Error preparing data for projected volume kube-api-access-zndx6 for pod openshift-network-diagnostics/network-check-target-g9ff4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:10:28.766467 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:28.766385 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6 podName:c769bcd4-a84b-488c-a780-2d5b36778839 nodeName:}" failed. No retries permitted until 2026-06-11 21:10:29.266368852 +0000 UTC m=+3.009448822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zndx6" (UniqueName: "kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6") pod "network-check-target-g9ff4" (UID: "c769bcd4-a84b-488c-a780-2d5b36778839") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:10:28.768709 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.768684 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86d5c\" (UniqueName: \"kubernetes.io/projected/7e44a0b9-6458-4700-8d71-d45f13e8ad3b-kube-api-access-86d5c\") pod \"multus-vtjdn\" (UID: \"7e44a0b9-6458-4700-8d71-d45f13e8ad3b\") " pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.768841 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.768798 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m45vc\" (UniqueName: \"kubernetes.io/projected/db4e8904-7461-4943-909f-6e9fe29dea61-kube-api-access-m45vc\") pod \"network-metrics-daemon-hz8r8\" (UID: \"db4e8904-7461-4943-909f-6e9fe29dea61\") " pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:28.769622 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.769601 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7xr2\" (UniqueName: \"kubernetes.io/projected/a76fe3c1-133a-4c0d-b36f-4f309a013579-kube-api-access-h7xr2\") pod \"node-resolver-9wcfm\" (UID: \"a76fe3c1-133a-4c0d-b36f-4f309a013579\") " pod="openshift-dns/node-resolver-9wcfm" Jun 11 21:10:28.769721 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.769653 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t584b\" (UniqueName: \"kubernetes.io/projected/23d74d65-50c4-45b2-bfb0-33bd224279c8-kube-api-access-t584b\") pod \"node-ca-vb7pc\" (UID: \"23d74d65-50c4-45b2-bfb0-33bd224279c8\") " pod="openshift-image-registry/node-ca-vb7pc" Jun 11 21:10:28.791456 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.791396 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 21:10:28.853283 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.853095 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mkwd5" Jun 11 21:10:28.860673 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.860646 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b6xn8" Jun 11 21:10:28.868455 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.868433 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:28.876013 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.875994 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" Jun 11 21:10:28.884577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.884551 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wb646" Jun 11 21:10:28.891211 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.891185 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vtjdn" Jun 11 21:10:28.897749 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.897728 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9wcfm" Jun 11 21:10:28.906356 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.906328 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l4pjg" Jun 11 21:10:28.911833 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:28.911814 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vb7pc" Jun 11 21:10:29.031178 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.031146 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 21:10:29.264850 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.264777 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:29.265035 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.264859 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs\") pod \"network-metrics-daemon-hz8r8\" (UID: \"db4e8904-7461-4943-909f-6e9fe29dea61\") " pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:29.265035 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:29.264963 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 21:10:29.265035 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:29.264986 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:10:29.265198 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:29.265068 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs podName:db4e8904-7461-4943-909f-6e9fe29dea61 nodeName:}" failed. No retries permitted until 2026-06-11 21:10:30.265049028 +0000 UTC m=+4.008128995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs") pod "network-metrics-daemon-hz8r8" (UID: "db4e8904-7461-4943-909f-6e9fe29dea61") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:10:29.265198 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:29.265091 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret podName:58f726ff-e37a-497f-8a7c-9216791bdd00 nodeName:}" failed. No retries permitted until 2026-06-11 21:10:30.265080482 +0000 UTC m=+4.008160466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret") pod "global-pull-secret-syncer-bqqv5" (UID: "58f726ff-e37a-497f-8a7c-9216791bdd00") : object "kube-system"/"original-pull-secret" not registered Jun 11 21:10:29.358731 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:29.358701 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38607a8f_f08d_4fce_a21a_c885cc677d51.slice/crio-0058a46c989866652c398d6819ecb271433fbeef45467d310cb1360a74da3f9a WatchSource:0}: Error finding container 0058a46c989866652c398d6819ecb271433fbeef45467d310cb1360a74da3f9a: Status 404 returned error can't find the container with id 0058a46c989866652c398d6819ecb271433fbeef45467d310cb1360a74da3f9a Jun 11 21:10:29.359733 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:29.359706 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca2f91ff_0e01_49e9_9bf1_3552f79b1f15.slice/crio-e708dced7a02ec94caa5a266b099a9eb996ff4a1634fe0fde576bcfc2362bafc WatchSource:0}: Error finding container e708dced7a02ec94caa5a266b099a9eb996ff4a1634fe0fde576bcfc2362bafc: Status 404 returned error can't find the container with id e708dced7a02ec94caa5a266b099a9eb996ff4a1634fe0fde576bcfc2362bafc Jun 11 21:10:29.363633 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:29.363607 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33f3fb6a_f938_457d_9905_b4e1d7b115a0.slice/crio-463e3bdef2aebfbd53ba135289cf6b8548c71f2681fb7cb407c9279e31dba099 WatchSource:0}: Error finding container 463e3bdef2aebfbd53ba135289cf6b8548c71f2681fb7cb407c9279e31dba099: Status 404 returned error can't find the container with id 463e3bdef2aebfbd53ba135289cf6b8548c71f2681fb7cb407c9279e31dba099 Jun 11 21:10:29.364296 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:29.364273 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49c4cbd2_e03d_4ab5_8ec3_deebf6a7374f.slice/crio-5577d4b7717aae7beff5fc12302310ebae53804efae829c06121704f79a4dbeb WatchSource:0}: Error finding container 5577d4b7717aae7beff5fc12302310ebae53804efae829c06121704f79a4dbeb: Status 404 returned error can't find the container with id 5577d4b7717aae7beff5fc12302310ebae53804efae829c06121704f79a4dbeb Jun 11 21:10:29.365182 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:29.365158 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaea1a453_0093_47c5_87bb_67c9fa59021c.slice/crio-02e6e2f996b7e693a7d6a708758db9856c87139f78372de35d84017c55bdc78e WatchSource:0}: Error finding container 02e6e2f996b7e693a7d6a708758db9856c87139f78372de35d84017c55bdc78e: Status 404 returned error can't find the container with id 02e6e2f996b7e693a7d6a708758db9856c87139f78372de35d84017c55bdc78e Jun 11 21:10:29.365182 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.365163 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zndx6\" (UniqueName: \"kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6\") pod \"network-check-target-g9ff4\" (UID: \"c769bcd4-a84b-488c-a780-2d5b36778839\") " pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:29.365323 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:29.365294 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 21:10:29.365323 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:29.365313 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 21:10:29.365521 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:29.365326 2563 projected.go:196] Error preparing data for projected volume kube-api-access-zndx6 for pod openshift-network-diagnostics/network-check-target-g9ff4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:10:29.365521 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:29.365375 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6 podName:c769bcd4-a84b-488c-a780-2d5b36778839 nodeName:}" failed. No retries permitted until 2026-06-11 21:10:30.365358999 +0000 UTC m=+4.108438968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zndx6" (UniqueName: "kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6") pod "network-check-target-g9ff4" (UID: "c769bcd4-a84b-488c-a780-2d5b36778839") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:10:29.365942 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:29.365879 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod933eb3d8_8042_429b_b122_83d4705c128c.slice/crio-1023abbc4d5ba32ce9b3d3bab7bf2c8bba8ca87b7b2a192697f1bd8eda199a4f WatchSource:0}: Error finding container 1023abbc4d5ba32ce9b3d3bab7bf2c8bba8ca87b7b2a192697f1bd8eda199a4f: Status 404 returned error can't find the container with id 1023abbc4d5ba32ce9b3d3bab7bf2c8bba8ca87b7b2a192697f1bd8eda199a4f Jun 11 21:10:29.367506 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:29.367482 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23d74d65_50c4_45b2_bfb0_33bd224279c8.slice/crio-95a064a594882d477ec83147c069de76613c75f584f71ffe2881c84409d0e328 WatchSource:0}: Error finding container 95a064a594882d477ec83147c069de76613c75f584f71ffe2881c84409d0e328: Status 404 returned error can't find the container with id 95a064a594882d477ec83147c069de76613c75f584f71ffe2881c84409d0e328 Jun 11 21:10:29.368339 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:29.368319 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e44a0b9_6458_4700_8d71_d45f13e8ad3b.slice/crio-e04e8ed2d30fa07d33fa7930c48f964fd0e899b3065daa0989ae334685577bff WatchSource:0}: Error finding container e04e8ed2d30fa07d33fa7930c48f964fd0e899b3065daa0989ae334685577bff: Status 404 returned error can't find the container with id e04e8ed2d30fa07d33fa7930c48f964fd0e899b3065daa0989ae334685577bff Jun 11 21:10:29.369848 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:10:29.369829 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda76fe3c1_133a_4c0d_b36f_4f309a013579.slice/crio-4eca7071dd03ed07c648a9942993122242f5c6d4a53632ade4047061cf3f12c5 WatchSource:0}: Error finding container 4eca7071dd03ed07c648a9942993122242f5c6d4a53632ade4047061cf3f12c5: Status 404 returned error can't find the container with id 4eca7071dd03ed07c648a9942993122242f5c6d4a53632ade4047061cf3f12c5 Jun 11 21:10:29.583555 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.583497 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 21:05:28 +0000 UTC" deadline="2028-01-10 01:34:52.935460895 +0000 UTC" Jun 11 21:10:29.583555 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.583553 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13852h24m23.351911315s" Jun 11 21:10:29.594955 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.594935 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 21:10:29.622229 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.622195 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l4pjg" event={"ID":"aea1a453-0093-47c5-87bb-67c9fa59021c","Type":"ContainerStarted","Data":"02e6e2f996b7e693a7d6a708758db9856c87139f78372de35d84017c55bdc78e"} Jun 11 21:10:29.623366 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.623341 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkwd5" event={"ID":"38607a8f-f08d-4fce-a21a-c885cc677d51","Type":"ContainerStarted","Data":"0058a46c989866652c398d6819ecb271433fbeef45467d310cb1360a74da3f9a"} Jun 11 21:10:29.625153 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.625130 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" event={"ID":"dc1c0b2bc241594d3e7f1c749bf01355","Type":"ContainerStarted","Data":"b3242fc5c12c02c6688702c4eef8cd2804bd083d6c421b757251247e43279e23"} Jun 11 21:10:29.626228 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.626207 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9wcfm" event={"ID":"a76fe3c1-133a-4c0d-b36f-4f309a013579","Type":"ContainerStarted","Data":"4eca7071dd03ed07c648a9942993122242f5c6d4a53632ade4047061cf3f12c5"} Jun 11 21:10:29.627248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.627216 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" event={"ID":"933eb3d8-8042-429b-b122-83d4705c128c","Type":"ContainerStarted","Data":"1023abbc4d5ba32ce9b3d3bab7bf2c8bba8ca87b7b2a192697f1bd8eda199a4f"} Jun 11 21:10:29.628328 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.628305 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" event={"ID":"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f","Type":"ContainerStarted","Data":"5577d4b7717aae7beff5fc12302310ebae53804efae829c06121704f79a4dbeb"} Jun 11 21:10:29.629347 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.629324 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b6xn8" event={"ID":"33f3fb6a-f938-457d-9905-b4e1d7b115a0","Type":"ContainerStarted","Data":"463e3bdef2aebfbd53ba135289cf6b8548c71f2681fb7cb407c9279e31dba099"} Jun 11 21:10:29.630342 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.630324 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wb646" event={"ID":"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15","Type":"ContainerStarted","Data":"e708dced7a02ec94caa5a266b099a9eb996ff4a1634fe0fde576bcfc2362bafc"} Jun 11 21:10:29.632547 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.632519 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal" event={"ID":"5b0fb3102504571ccdfcf8b539950afa","Type":"ContainerStarted","Data":"ee783dd1ecedc8251917f2b311c2e652418d9cd20412a7fcce68000095286186"} Jun 11 21:10:29.634147 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.634129 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vtjdn" event={"ID":"7e44a0b9-6458-4700-8d71-d45f13e8ad3b","Type":"ContainerStarted","Data":"e04e8ed2d30fa07d33fa7930c48f964fd0e899b3065daa0989ae334685577bff"} Jun 11 21:10:29.635081 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.635054 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vb7pc" event={"ID":"23d74d65-50c4-45b2-bfb0-33bd224279c8","Type":"ContainerStarted","Data":"95a064a594882d477ec83147c069de76613c75f584f71ffe2881c84409d0e328"} Jun 11 21:10:29.651713 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:29.651673 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-167.ec2.internal" podStartSLOduration=1.651660529 podStartE2EDuration="1.651660529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:10:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:10:29.651496589 +0000 UTC m=+3.394576570" watchObservedRunningTime="2026-06-11 21:10:29.651660529 +0000 UTC m=+3.394740517" Jun 11 21:10:30.272723 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:30.272674 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs\") pod \"network-metrics-daemon-hz8r8\" (UID: \"db4e8904-7461-4943-909f-6e9fe29dea61\") " pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:30.272914 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:30.272763 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:30.272914 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:30.272899 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 21:10:30.273053 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:30.272967 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret podName:58f726ff-e37a-497f-8a7c-9216791bdd00 nodeName:}" failed. No retries permitted until 2026-06-11 21:10:32.272947394 +0000 UTC m=+6.016027362 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret") pod "global-pull-secret-syncer-bqqv5" (UID: "58f726ff-e37a-497f-8a7c-9216791bdd00") : object "kube-system"/"original-pull-secret" not registered Jun 11 21:10:30.273596 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:30.273368 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:10:30.273596 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:30.273423 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs podName:db4e8904-7461-4943-909f-6e9fe29dea61 nodeName:}" failed. No retries permitted until 2026-06-11 21:10:32.273408323 +0000 UTC m=+6.016488292 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs") pod "network-metrics-daemon-hz8r8" (UID: "db4e8904-7461-4943-909f-6e9fe29dea61") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:10:30.374289 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:30.373661 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zndx6\" (UniqueName: \"kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6\") pod \"network-check-target-g9ff4\" (UID: \"c769bcd4-a84b-488c-a780-2d5b36778839\") " pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:30.374289 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:30.373826 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 21:10:30.374289 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:30.373844 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 21:10:30.374289 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:30.373858 2563 projected.go:196] Error preparing data for projected volume kube-api-access-zndx6 for pod openshift-network-diagnostics/network-check-target-g9ff4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:10:30.374289 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:30.373918 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6 podName:c769bcd4-a84b-488c-a780-2d5b36778839 nodeName:}" failed. No retries permitted until 2026-06-11 21:10:32.37389964 +0000 UTC m=+6.116979608 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zndx6" (UniqueName: "kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6") pod "network-check-target-g9ff4" (UID: "c769bcd4-a84b-488c-a780-2d5b36778839") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:10:30.584935 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:30.584571 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 21:05:28 +0000 UTC" deadline="2027-11-13 06:02:34.14678335 +0000 UTC" Jun 11 21:10:30.584935 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:30.584833 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12464h52m3.561956396s" Jun 11 21:10:30.614981 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:30.614066 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:30.614981 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:30.614200 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:30.614981 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:30.614596 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:30.614981 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:30.614702 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:30.614981 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:30.614764 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:30.614981 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:30.614868 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:30.670266 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:30.670227 2563 generic.go:356] "Generic (PLEG): container finished" podID="dc1c0b2bc241594d3e7f1c749bf01355" containerID="b3242fc5c12c02c6688702c4eef8cd2804bd083d6c421b757251247e43279e23" exitCode=0 Jun 11 21:10:30.671190 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:30.671162 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" event={"ID":"dc1c0b2bc241594d3e7f1c749bf01355","Type":"ContainerDied","Data":"b3242fc5c12c02c6688702c4eef8cd2804bd083d6c421b757251247e43279e23"} Jun 11 21:10:31.701272 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:31.700582 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" event={"ID":"dc1c0b2bc241594d3e7f1c749bf01355","Type":"ContainerStarted","Data":"84ca48352f12ad0830c34dde9213cb87135e96018b4a5a0857c71873dacad465"} Jun 11 21:10:31.719763 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:31.718692 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-167.ec2.internal" podStartSLOduration=3.718671014 podStartE2EDuration="3.718671014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:10:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:10:31.715360212 +0000 UTC m=+5.458440200" watchObservedRunningTime="2026-06-11 21:10:31.718671014 +0000 UTC m=+5.461751004" Jun 11 21:10:32.289094 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:32.289054 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:32.289277 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:32.289128 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs\") pod \"network-metrics-daemon-hz8r8\" (UID: \"db4e8904-7461-4943-909f-6e9fe29dea61\") " pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:32.289277 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:32.289254 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:10:32.289399 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:32.289316 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs podName:db4e8904-7461-4943-909f-6e9fe29dea61 nodeName:}" failed. No retries permitted until 2026-06-11 21:10:36.289298078 +0000 UTC m=+10.032378046 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs") pod "network-metrics-daemon-hz8r8" (UID: "db4e8904-7461-4943-909f-6e9fe29dea61") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:10:32.289755 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:32.289737 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 21:10:32.289830 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:32.289790 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret podName:58f726ff-e37a-497f-8a7c-9216791bdd00 nodeName:}" failed. No retries permitted until 2026-06-11 21:10:36.289775119 +0000 UTC m=+10.032855090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret") pod "global-pull-secret-syncer-bqqv5" (UID: "58f726ff-e37a-497f-8a7c-9216791bdd00") : object "kube-system"/"original-pull-secret" not registered Jun 11 21:10:32.389720 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:32.389679 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zndx6\" (UniqueName: \"kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6\") pod \"network-check-target-g9ff4\" (UID: \"c769bcd4-a84b-488c-a780-2d5b36778839\") " pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:32.389912 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:32.389895 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 21:10:32.389983 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:32.389921 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 21:10:32.389983 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:32.389935 2563 projected.go:196] Error preparing data for projected volume kube-api-access-zndx6 for pod openshift-network-diagnostics/network-check-target-g9ff4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:10:32.390121 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:32.389996 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6 podName:c769bcd4-a84b-488c-a780-2d5b36778839 nodeName:}" failed. No retries permitted until 2026-06-11 21:10:36.389977403 +0000 UTC m=+10.133057371 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zndx6" (UniqueName: "kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6") pod "network-check-target-g9ff4" (UID: "c769bcd4-a84b-488c-a780-2d5b36778839") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:10:32.612084 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:32.611371 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:32.612084 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:32.611371 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:32.612084 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:32.611501 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:32.612084 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:32.611603 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:32.612084 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:32.611721 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:32.612084 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:32.611820 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:34.612124 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:34.611544 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:34.612124 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:34.611757 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:34.612124 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:34.611856 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:34.612124 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:34.611977 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:34.612124 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:34.612081 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:34.612782 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:34.612187 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:36.324472 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:36.324050 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs\") pod \"network-metrics-daemon-hz8r8\" (UID: \"db4e8904-7461-4943-909f-6e9fe29dea61\") " pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:36.324472 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:36.324133 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:36.324472 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:36.324260 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 21:10:36.324472 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:36.324318 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret podName:58f726ff-e37a-497f-8a7c-9216791bdd00 nodeName:}" failed. No retries permitted until 2026-06-11 21:10:44.324299795 +0000 UTC m=+18.067379775 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret") pod "global-pull-secret-syncer-bqqv5" (UID: "58f726ff-e37a-497f-8a7c-9216791bdd00") : object "kube-system"/"original-pull-secret" not registered Jun 11 21:10:36.324472 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:36.324396 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:10:36.324472 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:36.324444 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs podName:db4e8904-7461-4943-909f-6e9fe29dea61 nodeName:}" failed. No retries permitted until 2026-06-11 21:10:44.324431494 +0000 UTC m=+18.067511478 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs") pod "network-metrics-daemon-hz8r8" (UID: "db4e8904-7461-4943-909f-6e9fe29dea61") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:10:36.425304 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:36.425267 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zndx6\" (UniqueName: \"kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6\") pod \"network-check-target-g9ff4\" (UID: \"c769bcd4-a84b-488c-a780-2d5b36778839\") " pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:36.425499 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:36.425469 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 21:10:36.425499 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:36.425488 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 21:10:36.425622 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:36.425502 2563 projected.go:196] Error preparing data for projected volume kube-api-access-zndx6 for pod openshift-network-diagnostics/network-check-target-g9ff4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:10:36.425622 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:36.425564 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6 podName:c769bcd4-a84b-488c-a780-2d5b36778839 nodeName:}" failed. No retries permitted until 2026-06-11 21:10:44.425544694 +0000 UTC m=+18.168624678 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zndx6" (UniqueName: "kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6") pod "network-check-target-g9ff4" (UID: "c769bcd4-a84b-488c-a780-2d5b36778839") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:10:36.613285 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:36.612286 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:36.613285 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:36.612421 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:36.613285 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:36.612455 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:36.613285 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:36.612538 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:36.613285 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:36.612581 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:36.613285 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:36.612635 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:38.611913 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:38.611883 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:38.612340 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:38.611883 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:38.612340 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:38.612040 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:38.612340 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:38.612078 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:38.612340 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:38.611883 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:38.612340 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:38.612164 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:40.614728 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:40.614646 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:40.615160 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:40.614646 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:40.615160 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:40.614757 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:40.615160 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:40.614842 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:40.615160 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:40.614645 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:40.615160 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:40.614912 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:42.611685 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:42.611648 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:42.612164 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:42.611652 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:42.612164 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:42.611784 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:42.612164 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:42.611889 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:42.612164 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:42.611654 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:42.612164 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:42.611984 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:44.384654 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:44.384568 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:44.385120 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:44.384727 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 21:10:44.385120 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:44.384739 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs\") pod \"network-metrics-daemon-hz8r8\" (UID: \"db4e8904-7461-4943-909f-6e9fe29dea61\") " pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:44.385120 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:44.384800 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret podName:58f726ff-e37a-497f-8a7c-9216791bdd00 nodeName:}" failed. No retries permitted until 2026-06-11 21:11:00.384779935 +0000 UTC m=+34.127859972 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret") pod "global-pull-secret-syncer-bqqv5" (UID: "58f726ff-e37a-497f-8a7c-9216791bdd00") : object "kube-system"/"original-pull-secret" not registered Jun 11 21:10:44.385120 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:44.384881 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:10:44.385120 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:44.384952 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs podName:db4e8904-7461-4943-909f-6e9fe29dea61 nodeName:}" failed. No retries permitted until 2026-06-11 21:11:00.384930674 +0000 UTC m=+34.128010669 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs") pod "network-metrics-daemon-hz8r8" (UID: "db4e8904-7461-4943-909f-6e9fe29dea61") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:10:44.485486 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:44.485450 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zndx6\" (UniqueName: \"kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6\") pod \"network-check-target-g9ff4\" (UID: \"c769bcd4-a84b-488c-a780-2d5b36778839\") " pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:44.485657 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:44.485640 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 21:10:44.485714 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:44.485666 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 21:10:44.485714 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:44.485680 2563 projected.go:196] Error preparing data for projected volume kube-api-access-zndx6 for pod openshift-network-diagnostics/network-check-target-g9ff4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:10:44.485806 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:44.485744 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6 podName:c769bcd4-a84b-488c-a780-2d5b36778839 nodeName:}" failed. No retries permitted until 2026-06-11 21:11:00.485725922 +0000 UTC m=+34.228805893 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zndx6" (UniqueName: "kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6") pod "network-check-target-g9ff4" (UID: "c769bcd4-a84b-488c-a780-2d5b36778839") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:10:44.614467 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:44.614408 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:44.614467 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:44.614437 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:44.614467 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:44.614437 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:44.614752 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:44.614542 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:44.614752 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:44.614608 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:44.614752 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:44.614698 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:46.612260 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:46.612223 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:46.612857 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:46.612375 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:46.612857 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:46.612395 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:46.612857 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:46.612483 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:46.612857 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:46.612532 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:46.612857 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:46.612651 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:48.614008 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.613649 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:48.614008 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:48.613933 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:48.614746 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.614322 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:48.614746 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:48.614411 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:48.614746 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.614674 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:48.614949 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:48.614771 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:48.738399 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.738368 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wb646" event={"ID":"ca2f91ff-0e01-49e9-9bf1-3552f79b1f15","Type":"ContainerStarted","Data":"b76ae4057d91390fbd308df1e53138d8992ba3fc92525c0a6ad2bb7b94f0a3a5"} Jun 11 21:10:48.739995 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.739964 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vtjdn" event={"ID":"7e44a0b9-6458-4700-8d71-d45f13e8ad3b","Type":"ContainerStarted","Data":"900c702b7ca49abaf4fd3a7bb19dcf0fcdebefa54208f851b4e13a2df7a3cfca"} Jun 11 21:10:48.741693 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.741661 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vb7pc" event={"ID":"23d74d65-50c4-45b2-bfb0-33bd224279c8","Type":"ContainerStarted","Data":"6e7c25cf5cea5b03e209abf87416bee4f50c32e073ac65402def78711fd4cc23"} Jun 11 21:10:48.743322 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.743293 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l4pjg" event={"ID":"aea1a453-0093-47c5-87bb-67c9fa59021c","Type":"ContainerStarted","Data":"292bb7fcbc62c42eaf905de4e10d672613c6400f1c9c482789e147ede151a0b2"} Jun 11 21:10:48.744925 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.744893 2563 generic.go:356] "Generic (PLEG): container finished" podID="38607a8f-f08d-4fce-a21a-c885cc677d51" containerID="889a051a7fa2dde5a2965c05ff6be6a9713f971945dc157ddc43f894062b9311" exitCode=0 Jun 11 21:10:48.745070 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.744939 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkwd5" event={"ID":"38607a8f-f08d-4fce-a21a-c885cc677d51","Type":"ContainerDied","Data":"889a051a7fa2dde5a2965c05ff6be6a9713f971945dc157ddc43f894062b9311"} Jun 11 21:10:48.746688 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.746658 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9wcfm" event={"ID":"a76fe3c1-133a-4c0d-b36f-4f309a013579","Type":"ContainerStarted","Data":"ebdf01cbe2229a947bb1ad770f7a2da7aa84c6e3504708e97866e8cef1aa1525"} Jun 11 21:10:48.748473 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.748399 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" event={"ID":"933eb3d8-8042-429b-b122-83d4705c128c","Type":"ContainerStarted","Data":"a5b04c342bf0ea22d61a094807f5d5231d7ee29e892618deb13f561acd5f3849"} Jun 11 21:10:48.750303 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.750283 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" event={"ID":"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f","Type":"ContainerStarted","Data":"a12c4e297e3cf362dc31503b32f14373feee181d5933db2279bfe01d8a026563"} Jun 11 21:10:48.750459 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.750337 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" event={"ID":"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f","Type":"ContainerStarted","Data":"804bbfbda8fc13e911d8472f7cc9c6f23a91f24df6162ffea4d6ad7fc2c96a5b"} Jun 11 21:10:48.756196 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.756152 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wb646" podStartSLOduration=3.739466084 podStartE2EDuration="22.756135115s" totalImagesPullingTime="19.016669031s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:10:48.755610574 +0000 UTC m=+22.498690562" watchObservedRunningTime="2026-06-11 21:10:48.756135115 +0000 UTC m=+22.499215102" Jun 11 21:10:48.785986 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.785947 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l4pjg" podStartSLOduration=8.220649769 podStartE2EDuration="22.785933169s" totalImagesPullingTime="14.5652834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:10:48.785812037 +0000 UTC m=+22.528892026" watchObservedRunningTime="2026-06-11 21:10:48.785933169 +0000 UTC m=+22.529013156" Jun 11 21:10:48.811770 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.811732 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vtjdn" podStartSLOduration=3.7584461019999997 podStartE2EDuration="22.811717639s" totalImagesPullingTime="19.053271537s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:10:48.811640377 +0000 UTC m=+22.554720401" watchObservedRunningTime="2026-06-11 21:10:48.811717639 +0000 UTC m=+22.554797627" Jun 11 21:10:48.811887 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.811846 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vb7pc" podStartSLOduration=3.8160249410000002 podStartE2EDuration="22.811842324s" totalImagesPullingTime="18.995817383s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:10:48.797632194 +0000 UTC m=+22.540712199" watchObservedRunningTime="2026-06-11 21:10:48.811842324 +0000 UTC m=+22.554922311" Jun 11 21:10:48.826134 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:48.825921 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9wcfm" podStartSLOduration=3.832197306 podStartE2EDuration="22.825908735s" totalImagesPullingTime="18.993711429s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:10:48.825807752 +0000 UTC m=+22.568887739" watchObservedRunningTime="2026-06-11 21:10:48.825908735 +0000 UTC m=+22.568988722" Jun 11 21:10:49.732439 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:49.732399 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l4pjg" Jun 11 21:10:49.732971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:49.732944 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l4pjg" Jun 11 21:10:49.757135 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:49.757103 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" event={"ID":"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f","Type":"ContainerStarted","Data":"5ec539b51916018706a26b6668844913cf7c5ec601b3e7a765cfdac84da06ff8"} Jun 11 21:10:49.757255 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:49.757199 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" event={"ID":"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f","Type":"ContainerStarted","Data":"905fced291173039736a0dc2c558392cd8a9bcce0d3846800f5a2c38050a5ba6"} Jun 11 21:10:49.757255 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:49.757251 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" event={"ID":"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f","Type":"ContainerStarted","Data":"8b8fc7d65756cd161232a68a5161697b24e0b7e9d0dc6ae89d07d5cafd3117ef"} Jun 11 21:10:49.757365 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:49.757303 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" event={"ID":"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f","Type":"ContainerStarted","Data":"50b070fded364160d0603a9a663297e9ba39202624419b342150ae41be4f09cb"} Jun 11 21:10:49.866203 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:49.866175 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 11 21:10:50.611425 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:50.611393 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:50.611713 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:50.611393 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:50.611713 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:50.611532 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:50.611713 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:50.611577 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:50.611713 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:50.611409 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:50.611713 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:50.611682 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:50.626794 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:50.626699 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-11T21:10:49.866196506Z","UUID":"1bb5def0-d927-43e8-a1da-66bf75a6b6be","Handler":null,"Name":"","Endpoint":""} Jun 11 21:10:50.628763 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:50.628742 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 11 21:10:50.628902 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:50.628783 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 11 21:10:50.761374 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:50.761336 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" event={"ID":"933eb3d8-8042-429b-b122-83d4705c128c","Type":"ContainerStarted","Data":"7e7f0aca33205c27a0b8618867d260179badd902bfd311a5946461d4071f64ba"} Jun 11 21:10:50.762982 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:50.762933 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b6xn8" event={"ID":"33f3fb6a-f938-457d-9905-b4e1d7b115a0","Type":"ContainerStarted","Data":"283a6fe0e10476678a4d7c1366e220c210bd45081326bf743a3808652b21cce3"} Jun 11 21:10:50.763141 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:50.762990 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 11 21:10:50.779832 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:50.779777 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b6xn8" podStartSLOduration=5.780233796 podStartE2EDuration="24.779757143s" totalImagesPullingTime="18.999523347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:10:50.778881053 +0000 UTC m=+24.521961053" watchObservedRunningTime="2026-06-11 21:10:50.779757143 +0000 UTC m=+24.522837164" Jun 11 21:10:51.767446 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:51.767409 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" event={"ID":"933eb3d8-8042-429b-b122-83d4705c128c","Type":"ContainerStarted","Data":"132e9d0292f1c88dedc17f1b36a3e935eff9c3940c29f43855604eba99b5834a"} Jun 11 21:10:51.771234 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:51.771206 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" event={"ID":"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f","Type":"ContainerStarted","Data":"adcc04e78c46772c1184543aac5964db5ec1f644f93d9126fe8f8f0b57003882"} Jun 11 21:10:51.784901 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:51.784851 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdn67" podStartSLOduration=4.727309967 podStartE2EDuration="25.784833578s" totalImagesPullingTime="21.057523611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:10:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:10:51.784656129 +0000 UTC m=+25.527736118" watchObservedRunningTime="2026-06-11 21:10:51.784833578 +0000 UTC m=+25.527913567" Jun 11 21:10:52.615190 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:52.615156 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:52.615348 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:52.615197 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:52.615348 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:52.615285 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:52.615348 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:52.615296 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:52.615486 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:52.615368 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:52.615486 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:52.615444 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:53.779974 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:53.779733 2563 generic.go:356] "Generic (PLEG): container finished" podID="38607a8f-f08d-4fce-a21a-c885cc677d51" containerID="78195ef85e1c4a775c6e7d601c00d6bea678b7de833916b46caa50881870e17c" exitCode=0 Jun 11 21:10:53.780555 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:53.779816 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkwd5" event={"ID":"38607a8f-f08d-4fce-a21a-c885cc677d51","Type":"ContainerDied","Data":"78195ef85e1c4a775c6e7d601c00d6bea678b7de833916b46caa50881870e17c"} Jun 11 21:10:53.783770 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:53.783743 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" event={"ID":"49c4cbd2-e03d-4ab5-8ec3-deebf6a7374f","Type":"ContainerStarted","Data":"603880f3116eb7a1c948fdace00c8c91719dcb884be052b77be0a4de47110dc0"} Jun 11 21:10:53.784060 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:53.784042 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:53.784151 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:53.784115 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:53.799785 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:53.799765 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:53.800229 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:53.800212 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:53.823420 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:53.823383 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" podStartSLOduration=8.759546256 podStartE2EDuration="27.823370823s" totalImagesPullingTime="19.063824567s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:10:53.822704683 +0000 UTC m=+27.565784710" watchObservedRunningTime="2026-06-11 21:10:53.823370823 +0000 UTC m=+27.566450811" Jun 11 21:10:54.614759 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:54.614732 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:54.614956 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:54.614831 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:54.614956 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:54.614856 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:54.614956 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:54.614861 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:54.614956 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:54.614932 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:54.615126 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:54.615014 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:54.788972 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:54.787853 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 11 21:10:55.727545 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:55.727386 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:10:55.792410 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:55.792382 2563 generic.go:356] "Generic (PLEG): container finished" podID="38607a8f-f08d-4fce-a21a-c885cc677d51" containerID="d991eb7eb1f9e9a6de4841bf797e7e7504a6a705b28600f0f69dcc31bb32154a" exitCode=0 Jun 11 21:10:55.792910 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:55.792468 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkwd5" event={"ID":"38607a8f-f08d-4fce-a21a-c885cc677d51","Type":"ContainerDied","Data":"d991eb7eb1f9e9a6de4841bf797e7e7504a6a705b28600f0f69dcc31bb32154a"} Jun 11 21:10:55.868789 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:55.868753 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g9ff4"] Jun 11 21:10:55.868975 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:55.868914 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:55.869067 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:55.869041 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:55.871609 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:55.871583 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bqqv5"] Jun 11 21:10:55.871734 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:55.871722 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:55.871821 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:55.871803 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:55.879441 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:55.879418 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hz8r8"] Jun 11 21:10:55.879533 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:55.879503 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:55.879602 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:55.879583 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:57.522479 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:57.522391 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l4pjg" Jun 11 21:10:57.523166 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:57.522549 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 11 21:10:57.523239 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:57.523184 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l4pjg" Jun 11 21:10:57.610879 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:57.610814 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:57.610879 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:57.610845 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:57.611156 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:57.610938 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4pjg_aea1a453-0093-47c5-87bb-67c9fa59021c/konnectivity-agent/0.log" limit=1024 Jun 11 21:10:57.611156 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:57.610943 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:10:57.611156 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:57.610988 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:57.611156 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:57.611052 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:57.611156 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:57.611139 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:57.699783 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:57.699751 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4pjg_aea1a453-0093-47c5-87bb-67c9fa59021c/konnectivity-agent/0.log" limit=1024 Jun 11 21:10:57.799445 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:57.799353 2563 generic.go:356] "Generic (PLEG): container finished" podID="38607a8f-f08d-4fce-a21a-c885cc677d51" containerID="b4c40322e6e6a860e16408647bb428e4fbac66bf65dc6bc12801e857873bfcb8" exitCode=0 Jun 11 21:10:57.799445 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:57.799412 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkwd5" event={"ID":"38607a8f-f08d-4fce-a21a-c885cc677d51","Type":"ContainerDied","Data":"b4c40322e6e6a860e16408647bb428e4fbac66bf65dc6bc12801e857873bfcb8"} Jun 11 21:10:59.611385 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:59.611348 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:10:59.611385 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:59.611387 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:10:59.612003 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:10:59.611352 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:10:59.612003 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:59.611481 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9ff4" podUID="c769bcd4-a84b-488c-a780-2d5b36778839" Jun 11 21:10:59.612003 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:59.611563 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqqv5" podUID="58f726ff-e37a-497f-8a7c-9216791bdd00" Jun 11 21:10:59.612003 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:10:59.611643 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:11:00.411319 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:00.411069 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:11:00.411522 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:00.411372 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs\") pod \"network-metrics-daemon-hz8r8\" (UID: \"db4e8904-7461-4943-909f-6e9fe29dea61\") " pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:11:00.411522 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:00.411235 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 21:11:00.411522 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:00.411458 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret podName:58f726ff-e37a-497f-8a7c-9216791bdd00 nodeName:}" failed. No retries permitted until 2026-06-11 21:11:32.411439256 +0000 UTC m=+66.154519234 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret") pod "global-pull-secret-syncer-bqqv5" (UID: "58f726ff-e37a-497f-8a7c-9216791bdd00") : object "kube-system"/"original-pull-secret" not registered Jun 11 21:11:00.411522 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:00.411502 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:11:00.411754 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:00.411540 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs podName:db4e8904-7461-4943-909f-6e9fe29dea61 nodeName:}" failed. No retries permitted until 2026-06-11 21:11:32.411527008 +0000 UTC m=+66.154606974 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs") pod "network-metrics-daemon-hz8r8" (UID: "db4e8904-7461-4943-909f-6e9fe29dea61") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:11:00.512181 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:00.512146 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zndx6\" (UniqueName: \"kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6\") pod \"network-check-target-g9ff4\" (UID: \"c769bcd4-a84b-488c-a780-2d5b36778839\") " pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:11:00.512349 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:00.512309 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 21:11:00.512349 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:00.512328 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 21:11:00.512349 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:00.512338 2563 projected.go:196] Error preparing data for projected volume kube-api-access-zndx6 for pod openshift-network-diagnostics/network-check-target-g9ff4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:11:00.512462 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:00.512388 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6 podName:c769bcd4-a84b-488c-a780-2d5b36778839 nodeName:}" failed. No retries permitted until 2026-06-11 21:11:32.512374253 +0000 UTC m=+66.255454218 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zndx6" (UniqueName: "kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6") pod "network-check-target-g9ff4" (UID: "c769bcd4-a84b-488c-a780-2d5b36778839") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:11:01.079113 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.079085 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-167.ec2.internal" event="NodeReady" Jun 11 21:11:01.079478 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.079237 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 11 21:11:01.118410 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.118378 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5qxs8"] Jun 11 21:11:01.123801 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.123775 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zvmqc"] Jun 11 21:11:01.123954 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.123916 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:01.127523 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.127501 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zvmqc" Jun 11 21:11:01.128700 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.128672 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 11 21:11:01.128826 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.128700 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6v9mc\"" Jun 11 21:11:01.128826 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.128674 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 11 21:11:01.129795 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.129742 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 11 21:11:01.129795 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.129764 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 11 21:11:01.129956 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.129802 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rttxf\"" Jun 11 21:11:01.129956 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.129815 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 11 21:11:01.133012 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.132991 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zvmqc"] Jun 11 21:11:01.139387 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.139366 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5qxs8"] Jun 11 21:11:01.217370 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.217337 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:01.217370 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.217377 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm66l\" (UniqueName: \"kubernetes.io/projected/faf3cd10-7238-4409-bf88-932cd0b1c14e-kube-api-access-jm66l\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:01.217625 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.217407 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/faf3cd10-7238-4409-bf88-932cd0b1c14e-config-volume\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:01.217625 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.217475 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6klg9\" (UniqueName: \"kubernetes.io/projected/99fddcb2-b296-415e-94a9-15a40dc7462d-kube-api-access-6klg9\") pod \"ingress-canary-zvmqc\" (UID: \"99fddcb2-b296-415e-94a9-15a40dc7462d\") " pod="openshift-ingress-canary/ingress-canary-zvmqc" Jun 11 21:11:01.217625 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.217519 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99fddcb2-b296-415e-94a9-15a40dc7462d-cert\") pod \"ingress-canary-zvmqc\" (UID: \"99fddcb2-b296-415e-94a9-15a40dc7462d\") " pod="openshift-ingress-canary/ingress-canary-zvmqc" Jun 11 21:11:01.217766 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.217634 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faf3cd10-7238-4409-bf88-932cd0b1c14e-tmp-dir\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:01.318695 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.318658 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faf3cd10-7238-4409-bf88-932cd0b1c14e-tmp-dir\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:01.318695 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.318698 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:01.318935 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.318724 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm66l\" (UniqueName: \"kubernetes.io/projected/faf3cd10-7238-4409-bf88-932cd0b1c14e-kube-api-access-jm66l\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:01.318935 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:01.318845 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:11:01.318935 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.318873 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/faf3cd10-7238-4409-bf88-932cd0b1c14e-config-volume\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:01.318935 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:01.318915 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls podName:faf3cd10-7238-4409-bf88-932cd0b1c14e nodeName:}" failed. No retries permitted until 2026-06-11 21:11:01.818893838 +0000 UTC m=+35.561973804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls") pod "dns-default-5qxs8" (UID: "faf3cd10-7238-4409-bf88-932cd0b1c14e") : secret "dns-default-metrics-tls" not found Jun 11 21:11:01.319187 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.318956 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6klg9\" (UniqueName: \"kubernetes.io/projected/99fddcb2-b296-415e-94a9-15a40dc7462d-kube-api-access-6klg9\") pod \"ingress-canary-zvmqc\" (UID: \"99fddcb2-b296-415e-94a9-15a40dc7462d\") " pod="openshift-ingress-canary/ingress-canary-zvmqc" Jun 11 21:11:01.319187 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.319000 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99fddcb2-b296-415e-94a9-15a40dc7462d-cert\") pod \"ingress-canary-zvmqc\" (UID: \"99fddcb2-b296-415e-94a9-15a40dc7462d\") " pod="openshift-ingress-canary/ingress-canary-zvmqc" Jun 11 21:11:01.319187 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.319106 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faf3cd10-7238-4409-bf88-932cd0b1c14e-tmp-dir\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:01.319470 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.319445 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/faf3cd10-7238-4409-bf88-932cd0b1c14e-config-volume\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:01.323998 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.323975 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99fddcb2-b296-415e-94a9-15a40dc7462d-cert\") pod \"ingress-canary-zvmqc\" (UID: \"99fddcb2-b296-415e-94a9-15a40dc7462d\") " pod="openshift-ingress-canary/ingress-canary-zvmqc" Jun 11 21:11:01.327884 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.327864 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm66l\" (UniqueName: \"kubernetes.io/projected/faf3cd10-7238-4409-bf88-932cd0b1c14e-kube-api-access-jm66l\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:01.328163 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.328142 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6klg9\" (UniqueName: \"kubernetes.io/projected/99fddcb2-b296-415e-94a9-15a40dc7462d-kube-api-access-6klg9\") pod \"ingress-canary-zvmqc\" (UID: \"99fddcb2-b296-415e-94a9-15a40dc7462d\") " pod="openshift-ingress-canary/ingress-canary-zvmqc" Jun 11 21:11:01.444200 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.444108 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zvmqc" Jun 11 21:11:01.611798 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.611755 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:11:01.611798 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.611800 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:11:01.612053 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.611874 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:11:01.614251 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.614221 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 21:11:01.614371 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.614256 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-thzvh\"" Jun 11 21:11:01.615196 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.615174 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 21:11:01.615317 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.615282 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9kn59\"" Jun 11 21:11:01.615518 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.615486 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 21:11:01.615623 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.615562 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 21:11:01.822031 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.821988 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:01.822272 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:01.822136 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:11:01.822272 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:01.822217 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls podName:faf3cd10-7238-4409-bf88-932cd0b1c14e nodeName:}" failed. No retries permitted until 2026-06-11 21:11:02.82219957 +0000 UTC m=+36.565279550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls") pod "dns-default-5qxs8" (UID: "faf3cd10-7238-4409-bf88-932cd0b1c14e") : secret "dns-default-metrics-tls" not found Jun 11 21:11:01.894404 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.894372 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4pjg_aea1a453-0093-47c5-87bb-67c9fa59021c/konnectivity-agent/0.log" limit=1024 Jun 11 21:11:01.951001 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:01.950965 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4pjg_aea1a453-0093-47c5-87bb-67c9fa59021c/konnectivity-agent/0.log" limit=1024 Jun 11 21:11:02.681354 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:02.681327 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zvmqc"] Jun 11 21:11:02.684751 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:11:02.684721 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99fddcb2_b296_415e_94a9_15a40dc7462d.slice/crio-a14ed6fac50d42fb223594bd4fb5aa29dc47e026a3d99fb1076da206cbbc4fce WatchSource:0}: Error finding container a14ed6fac50d42fb223594bd4fb5aa29dc47e026a3d99fb1076da206cbbc4fce: Status 404 returned error can't find the container with id a14ed6fac50d42fb223594bd4fb5aa29dc47e026a3d99fb1076da206cbbc4fce Jun 11 21:11:02.814110 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:02.813910 2563 generic.go:356] "Generic (PLEG): container finished" podID="38607a8f-f08d-4fce-a21a-c885cc677d51" containerID="fa4722c19eba8cf2bb479de0a81524236c6381e377471bfd7a59265b7cce52c0" exitCode=0 Jun 11 21:11:02.814234 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:02.813986 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkwd5" event={"ID":"38607a8f-f08d-4fce-a21a-c885cc677d51","Type":"ContainerDied","Data":"fa4722c19eba8cf2bb479de0a81524236c6381e377471bfd7a59265b7cce52c0"} Jun 11 21:11:02.815328 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:02.815307 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zvmqc" event={"ID":"99fddcb2-b296-415e-94a9-15a40dc7462d","Type":"ContainerStarted","Data":"a14ed6fac50d42fb223594bd4fb5aa29dc47e026a3d99fb1076da206cbbc4fce"} Jun 11 21:11:02.830038 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:02.829999 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:02.830136 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:02.830119 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:11:02.830183 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:02.830170 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls podName:faf3cd10-7238-4409-bf88-932cd0b1c14e nodeName:}" failed. No retries permitted until 2026-06-11 21:11:04.830157001 +0000 UTC m=+38.573236966 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls") pod "dns-default-5qxs8" (UID: "faf3cd10-7238-4409-bf88-932cd0b1c14e") : secret "dns-default-metrics-tls" not found Jun 11 21:11:03.822107 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:03.822069 2563 generic.go:356] "Generic (PLEG): container finished" podID="38607a8f-f08d-4fce-a21a-c885cc677d51" containerID="5a0bafd4ef933c6edd476eca7c17812b99fdfa1145c027baf868c26dd9a0cfc2" exitCode=0 Jun 11 21:11:03.822535 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:03.822148 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkwd5" event={"ID":"38607a8f-f08d-4fce-a21a-c885cc677d51","Type":"ContainerDied","Data":"5a0bafd4ef933c6edd476eca7c17812b99fdfa1145c027baf868c26dd9a0cfc2"} Jun 11 21:11:04.827142 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:04.827106 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkwd5" event={"ID":"38607a8f-f08d-4fce-a21a-c885cc677d51","Type":"ContainerStarted","Data":"c2675927210799b3f20c08d9ad14bc2660f9089f67e13e55af329afb3a147e0b"} Jun 11 21:11:04.828509 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:04.828488 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zvmqc" event={"ID":"99fddcb2-b296-415e-94a9-15a40dc7462d","Type":"ContainerStarted","Data":"a66ff15e52363373008863be0e408e1b48a6535c6f8980c2bf43aa97e5738f39"} Jun 11 21:11:04.847038 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:04.847000 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:04.847161 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:04.847144 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:11:04.847209 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:04.847200 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls podName:faf3cd10-7238-4409-bf88-932cd0b1c14e nodeName:}" failed. No retries permitted until 2026-06-11 21:11:08.847187036 +0000 UTC m=+42.590267002 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls") pod "dns-default-5qxs8" (UID: "faf3cd10-7238-4409-bf88-932cd0b1c14e") : secret "dns-default-metrics-tls" not found Jun 11 21:11:04.850506 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:04.850467 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mkwd5" podStartSLOduration=7.817445271 podStartE2EDuration="38.85045226s" totalImagesPullingTime="31.033006989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:10:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:11:04.848850187 +0000 UTC m=+38.591930186" watchObservedRunningTime="2026-06-11 21:11:04.85045226 +0000 UTC m=+38.593532247" Jun 11 21:11:04.862526 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:04.862485 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zvmqc" podStartSLOduration=2.149949069 podStartE2EDuration="3.862472969s" totalImagesPullingTime="1.7125239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:11:04.861795938 +0000 UTC m=+38.604875926" watchObservedRunningTime="2026-06-11 21:11:04.862472969 +0000 UTC m=+38.605552957" Jun 11 21:11:04.975437 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:04.975393 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4pjg_aea1a453-0093-47c5-87bb-67c9fa59021c/konnectivity-agent/0.log" limit=1024 Jun 11 21:11:05.028688 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:05.028652 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4pjg_aea1a453-0093-47c5-87bb-67c9fa59021c/konnectivity-agent/0.log" limit=1024 Jun 11 21:11:05.559798 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:05.559766 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4pjg_aea1a453-0093-47c5-87bb-67c9fa59021c/konnectivity-agent/0.log" limit=1024 Jun 11 21:11:08.877128 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:08.877096 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:08.877712 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:08.877229 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:11:08.877712 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:08.877301 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls podName:faf3cd10-7238-4409-bf88-932cd0b1c14e nodeName:}" failed. No retries permitted until 2026-06-11 21:11:16.87728167 +0000 UTC m=+50.620361636 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls") pod "dns-default-5qxs8" (UID: "faf3cd10-7238-4409-bf88-932cd0b1c14e") : secret "dns-default-metrics-tls" not found Jun 11 21:11:09.545749 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:09.545706 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4pjg_aea1a453-0093-47c5-87bb-67c9fa59021c/konnectivity-agent/0.log" limit=1024 Jun 11 21:11:16.933909 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:16.933872 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:16.934463 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:16.934039 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:11:16.934463 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:16.934104 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls podName:faf3cd10-7238-4409-bf88-932cd0b1c14e nodeName:}" failed. No retries permitted until 2026-06-11 21:11:32.934088357 +0000 UTC m=+66.677168323 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls") pod "dns-default-5qxs8" (UID: "faf3cd10-7238-4409-bf88-932cd0b1c14e") : secret "dns-default-metrics-tls" not found Jun 11 21:11:26.814834 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:26.814804 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhxkp" Jun 11 21:11:32.432283 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.432238 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:11:32.432283 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.432293 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs\") pod \"network-metrics-daemon-hz8r8\" (UID: \"db4e8904-7461-4943-909f-6e9fe29dea61\") " pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:11:32.435112 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.435092 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 21:11:32.435180 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.435125 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 21:11:32.443232 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:32.443220 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 11 21:11:32.443288 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:32.443278 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs podName:db4e8904-7461-4943-909f-6e9fe29dea61 nodeName:}" failed. No retries permitted until 2026-06-11 21:12:36.443262881 +0000 UTC m=+130.186342850 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs") pod "network-metrics-daemon-hz8r8" (UID: "db4e8904-7461-4943-909f-6e9fe29dea61") : secret "metrics-daemon-secret" not found Jun 11 21:11:32.446448 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.446429 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/58f726ff-e37a-497f-8a7c-9216791bdd00-original-pull-secret\") pod \"global-pull-secret-syncer-bqqv5\" (UID: \"58f726ff-e37a-497f-8a7c-9216791bdd00\") " pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:11:32.533709 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.533671 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zndx6\" (UniqueName: \"kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6\") pod \"network-check-target-g9ff4\" (UID: \"c769bcd4-a84b-488c-a780-2d5b36778839\") " pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:11:32.536471 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.536451 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqqv5" Jun 11 21:11:32.536527 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.536483 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 21:11:32.546993 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.546977 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 21:11:32.557362 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.557339 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zndx6\" (UniqueName: \"kubernetes.io/projected/c769bcd4-a84b-488c-a780-2d5b36778839-kube-api-access-zndx6\") pod \"network-check-target-g9ff4\" (UID: \"c769bcd4-a84b-488c-a780-2d5b36778839\") " pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:11:32.655328 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.655300 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bqqv5"] Jun 11 21:11:32.658822 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:11:32.658796 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58f726ff_e37a_497f_8a7c_9216791bdd00.slice/crio-0a79a3bf7db0f214448c20080435ca24d074ee78823e3cc6639c828f4d0c354f WatchSource:0}: Error finding container 0a79a3bf7db0f214448c20080435ca24d074ee78823e3cc6639c828f4d0c354f: Status 404 returned error can't find the container with id 0a79a3bf7db0f214448c20080435ca24d074ee78823e3cc6639c828f4d0c354f Jun 11 21:11:32.833330 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.833299 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9kn59\"" Jun 11 21:11:32.841313 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.841290 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:11:32.893847 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.893772 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bqqv5" event={"ID":"58f726ff-e37a-497f-8a7c-9216791bdd00","Type":"ContainerStarted","Data":"0a79a3bf7db0f214448c20080435ca24d074ee78823e3cc6639c828f4d0c354f"} Jun 11 21:11:32.939891 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.937323 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:11:32.939891 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:32.937514 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:11:32.939891 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:11:32.937586 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls podName:faf3cd10-7238-4409-bf88-932cd0b1c14e nodeName:}" failed. No retries permitted until 2026-06-11 21:12:04.937566098 +0000 UTC m=+98.680646079 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls") pod "dns-default-5qxs8" (UID: "faf3cd10-7238-4409-bf88-932cd0b1c14e") : secret "dns-default-metrics-tls" not found Jun 11 21:11:32.975174 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:32.975142 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g9ff4"] Jun 11 21:11:32.977846 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:11:32.977824 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc769bcd4_a84b_488c_a780_2d5b36778839.slice/crio-fad339069c7e6438056b5ab1b53c00b91d21b3cbfb151a51d6e641d1c6935b98 WatchSource:0}: Error finding container fad339069c7e6438056b5ab1b53c00b91d21b3cbfb151a51d6e641d1c6935b98: Status 404 returned error can't find the container with id fad339069c7e6438056b5ab1b53c00b91d21b3cbfb151a51d6e641d1c6935b98 Jun 11 21:11:33.897553 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:33.897514 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g9ff4" event={"ID":"c769bcd4-a84b-488c-a780-2d5b36778839","Type":"ContainerStarted","Data":"fad339069c7e6438056b5ab1b53c00b91d21b3cbfb151a51d6e641d1c6935b98"} Jun 11 21:11:34.852561 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:34.852522 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq"] Jun 11 21:11:34.884161 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:34.884122 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq"] Jun 11 21:11:34.884345 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:34.884234 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq" Jun 11 21:11:34.886884 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:34.886861 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 11 21:11:34.886884 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:34.886877 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 11 21:11:34.887922 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:34.887893 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 11 21:11:34.888161 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:34.888144 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 11 21:11:34.888314 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:34.888285 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jhxtk\"" Jun 11 21:11:34.952272 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:34.952249 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/40ccfc83-c00b-42d2-b7d3-ce9818e9c686-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8659d4b54d-rt7fq\" (UID: \"40ccfc83-c00b-42d2-b7d3-ce9818e9c686\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq" Jun 11 21:11:34.952592 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:34.952292 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkjml\" (UniqueName: \"kubernetes.io/projected/40ccfc83-c00b-42d2-b7d3-ce9818e9c686-kube-api-access-dkjml\") pod \"managed-serviceaccount-addon-agent-8659d4b54d-rt7fq\" (UID: \"40ccfc83-c00b-42d2-b7d3-ce9818e9c686\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq" Jun 11 21:11:35.053639 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:35.053593 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/40ccfc83-c00b-42d2-b7d3-ce9818e9c686-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8659d4b54d-rt7fq\" (UID: \"40ccfc83-c00b-42d2-b7d3-ce9818e9c686\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq" Jun 11 21:11:35.053864 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:35.053650 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkjml\" (UniqueName: \"kubernetes.io/projected/40ccfc83-c00b-42d2-b7d3-ce9818e9c686-kube-api-access-dkjml\") pod \"managed-serviceaccount-addon-agent-8659d4b54d-rt7fq\" (UID: \"40ccfc83-c00b-42d2-b7d3-ce9818e9c686\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq" Jun 11 21:11:35.056425 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:35.056399 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/40ccfc83-c00b-42d2-b7d3-ce9818e9c686-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8659d4b54d-rt7fq\" (UID: \"40ccfc83-c00b-42d2-b7d3-ce9818e9c686\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq" Jun 11 21:11:35.063401 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:35.063368 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkjml\" (UniqueName: \"kubernetes.io/projected/40ccfc83-c00b-42d2-b7d3-ce9818e9c686-kube-api-access-dkjml\") pod \"managed-serviceaccount-addon-agent-8659d4b54d-rt7fq\" (UID: \"40ccfc83-c00b-42d2-b7d3-ce9818e9c686\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq" Jun 11 21:11:35.208044 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:35.207949 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq" Jun 11 21:11:35.916454 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:35.916424 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq"] Jun 11 21:11:35.920368 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:11:35.920347 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40ccfc83_c00b_42d2_b7d3_ce9818e9c686.slice/crio-8f1e81cc65b94ea2ae2ea7a2011b03ab0bda14ffab61707bb05611c363ed0ab3 WatchSource:0}: Error finding container 8f1e81cc65b94ea2ae2ea7a2011b03ab0bda14ffab61707bb05611c363ed0ab3: Status 404 returned error can't find the container with id 8f1e81cc65b94ea2ae2ea7a2011b03ab0bda14ffab61707bb05611c363ed0ab3 Jun 11 21:11:36.911115 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:36.910821 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g9ff4" event={"ID":"c769bcd4-a84b-488c-a780-2d5b36778839","Type":"ContainerStarted","Data":"632566a11ef7304ab6f5e8e78bf61177a080a22452679290a38fc87bf1135dd5"} Jun 11 21:11:36.911115 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:36.910925 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:11:36.912464 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:36.912424 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq" event={"ID":"40ccfc83-c00b-42d2-b7d3-ce9818e9c686","Type":"ContainerStarted","Data":"8f1e81cc65b94ea2ae2ea7a2011b03ab0bda14ffab61707bb05611c363ed0ab3"} Jun 11 21:11:36.927310 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:36.927179 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-g9ff4" podStartSLOduration=68.061714561 podStartE2EDuration="1m10.927159714s" totalImagesPullingTime="2.865445153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:11:36.926763359 +0000 UTC m=+70.669843364" watchObservedRunningTime="2026-06-11 21:11:36.927159714 +0000 UTC m=+70.670239702" Jun 11 21:11:39.922159 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:39.922125 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bqqv5" event={"ID":"58f726ff-e37a-497f-8a7c-9216791bdd00","Type":"ContainerStarted","Data":"16e7b72f6d85e15a9720572c2e7a7d7247afc013d5d8418da038ee2de8aeaa57"} Jun 11 21:11:39.923475 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:39.923451 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq" event={"ID":"40ccfc83-c00b-42d2-b7d3-ce9818e9c686","Type":"ContainerStarted","Data":"9e3c939220d2fb8ed24bcfcb846f67a36af4a650dd405d3a307beed83159c092"} Jun 11 21:11:39.937682 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:39.937643 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bqqv5" podStartSLOduration=67.35414818 podStartE2EDuration="1m13.937628357s" totalImagesPullingTime="6.583480177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:11:39.937523651 +0000 UTC m=+73.680603648" watchObservedRunningTime="2026-06-11 21:11:39.937628357 +0000 UTC m=+73.680708322" Jun 11 21:11:39.950642 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:11:39.950598 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq" podStartSLOduration=2.584961604 podStartE2EDuration="5.950580618s" totalImagesPullingTime="3.365619014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:11:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:11:39.950486 +0000 UTC m=+73.693565986" watchObservedRunningTime="2026-06-11 21:11:39.950580618 +0000 UTC m=+73.693660603" Jun 11 21:12:04.956583 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:12:04.956547 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:12:04.956954 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:12:04.956659 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:12:04.956954 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:12:04.956712 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls podName:faf3cd10-7238-4409-bf88-932cd0b1c14e nodeName:}" failed. No retries permitted until 2026-06-11 21:13:08.956698461 +0000 UTC m=+162.699778426 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls") pod "dns-default-5qxs8" (UID: "faf3cd10-7238-4409-bf88-932cd0b1c14e") : secret "dns-default-metrics-tls" not found Jun 11 21:12:07.917513 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:12:07.917483 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-g9ff4" Jun 11 21:12:36.462672 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:12:36.462633 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs\") pod \"network-metrics-daemon-hz8r8\" (UID: \"db4e8904-7461-4943-909f-6e9fe29dea61\") " pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:12:36.463292 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:12:36.462776 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 11 21:12:36.463292 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:12:36.462905 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs podName:db4e8904-7461-4943-909f-6e9fe29dea61 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:38.462883303 +0000 UTC m=+252.205963282 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs") pod "network-metrics-daemon-hz8r8" (UID: "db4e8904-7461-4943-909f-6e9fe29dea61") : secret "metrics-daemon-secret" not found Jun 11 21:13:04.138754 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:13:04.138702 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5qxs8" podUID="faf3cd10-7238-4409-bf88-932cd0b1c14e" Jun 11 21:13:04.624629 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:13:04.624599 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-hz8r8" podUID="db4e8904-7461-4943-909f-6e9fe29dea61" Jun 11 21:13:05.122405 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:13:05.122381 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5qxs8" Jun 11 21:13:08.979518 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:13:08.979481 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:13:08.979912 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:13:08.979608 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:13:08.979912 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:13:08.979670 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls podName:faf3cd10-7238-4409-bf88-932cd0b1c14e nodeName:}" failed. No retries permitted until 2026-06-11 21:15:10.979655497 +0000 UTC m=+284.722735462 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls") pod "dns-default-5qxs8" (UID: "faf3cd10-7238-4409-bf88-932cd0b1c14e") : secret "dns-default-metrics-tls" not found Jun 11 21:13:19.611807 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:13:19.611722 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:13:40.205271 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:13:40.205232 2563 generic.go:356] "Generic (PLEG): container finished" podID="40ccfc83-c00b-42d2-b7d3-ce9818e9c686" containerID="9e3c939220d2fb8ed24bcfcb846f67a36af4a650dd405d3a307beed83159c092" exitCode=255 Jun 11 21:13:40.205733 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:13:40.205304 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq" event={"ID":"40ccfc83-c00b-42d2-b7d3-ce9818e9c686","Type":"ContainerDied","Data":"9e3c939220d2fb8ed24bcfcb846f67a36af4a650dd405d3a307beed83159c092"} Jun 11 21:13:40.210812 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:13:40.210795 2563 scope.go:117] "RemoveContainer" containerID="9e3c939220d2fb8ed24bcfcb846f67a36af4a650dd405d3a307beed83159c092" Jun 11 21:13:41.211912 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:13:41.211874 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8659d4b54d-rt7fq" event={"ID":"40ccfc83-c00b-42d2-b7d3-ce9818e9c686","Type":"ContainerStarted","Data":"203766d2bef91e7fefaaf802bca1b8758f764d34a5aeb8e66cc8c2d7e513609d"} Jun 11 21:14:02.428850 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.428819 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7988c46b7d-xgkmt"] Jun 11 21:14:02.431571 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.431555 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5d798db97b-wnl55"] Jun 11 21:14:02.431696 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.431677 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.434408 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.434378 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:02.435774 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.435753 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 11 21:14:02.435926 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.435915 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5qcb7\"" Jun 11 21:14:02.440264 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.440244 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 11 21:14:02.440378 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.440289 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 11 21:14:02.440378 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.440289 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 11 21:14:02.440533 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.440519 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 11 21:14:02.440631 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.440613 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 11 21:14:02.441163 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.441146 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 11 21:14:02.441453 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.441434 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 11 21:14:02.441453 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.441448 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-v4wq7\"" Jun 11 21:14:02.441586 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.441449 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 11 21:14:02.445451 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.445433 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 11 21:14:02.453704 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.453661 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d798db97b-wnl55"] Jun 11 21:14:02.454541 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.454524 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7988c46b7d-xgkmt"] Jun 11 21:14:02.530914 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.530877 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5dbbb5d744-tnsxc"] Jun 11 21:14:02.533135 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.533112 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7f6b62f4-15c4-468b-9858-cfbcf2518e55-ca-trust-extracted\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.533214 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.533142 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-certificates\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.533214 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.533163 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj4k4\" (UniqueName: \"kubernetes.io/projected/e4baf798-247a-4237-a103-aa1dae1b1557-kube-api-access-pj4k4\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:02.533214 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.533186 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f6b62f4-15c4-468b-9858-cfbcf2518e55-trusted-ca\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.533333 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.533223 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-default-certificate\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:02.533333 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.533239 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:02.533333 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.533264 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7f6b62f4-15c4-468b-9858-cfbcf2518e55-installation-pull-secrets\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.533333 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.533278 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-bound-sa-token\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.533333 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.533298 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htxsl\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-kube-api-access-htxsl\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.533333 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.533332 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:02.533541 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.533365 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.533541 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.533383 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7f6b62f4-15c4-468b-9858-cfbcf2518e55-image-registry-private-configuration\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.533541 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.533398 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-stats-auth\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:02.533795 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.533781 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:02.536374 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.536344 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 11 21:14:02.536374 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.536369 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-tp76j\"" Jun 11 21:14:02.536910 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.536886 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 11 21:14:02.537370 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.537352 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 11 21:14:02.537465 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.537410 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 11 21:14:02.542030 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.541998 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 11 21:14:02.546002 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.545980 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5dbbb5d744-tnsxc"] Jun 11 21:14:02.634411 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.634382 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7f6b62f4-15c4-468b-9858-cfbcf2518e55-installation-pull-secrets\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.634411 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.634415 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-bound-sa-token\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.634608 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.634435 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htxsl\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-kube-api-access-htxsl\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.634641 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.634618 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:02.634695 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.634678 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.634737 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.634719 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7f6b62f4-15c4-468b-9858-cfbcf2518e55-image-registry-private-configuration\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.634776 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:02.634751 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 21:14:02.634776 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.634753 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-stats-auth\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:02.634870 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:02.634798 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs podName:e4baf798-247a-4237-a103-aa1dae1b1557 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:03.134782626 +0000 UTC m=+216.877862592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs") pod "router-default-5d798db97b-wnl55" (UID: "e4baf798-247a-4237-a103-aa1dae1b1557") : secret "router-metrics-certs-default" not found Jun 11 21:14:02.634870 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.634815 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7f6b62f4-15c4-468b-9858-cfbcf2518e55-ca-trust-extracted\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.634870 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.634841 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-certificates\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.634870 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.634866 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj4k4\" (UniqueName: \"kubernetes.io/projected/e4baf798-247a-4237-a103-aa1dae1b1557-kube-api-access-pj4k4\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:02.635103 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:02.634843 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 21:14:02.635103 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:02.635001 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7988c46b7d-xgkmt: secret "image-registry-tls" not found Jun 11 21:14:02.635103 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:02.635067 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls podName:7f6b62f4-15c4-468b-9858-cfbcf2518e55 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:03.135051868 +0000 UTC m=+216.878131853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls") pod "image-registry-7988c46b7d-xgkmt" (UID: "7f6b62f4-15c4-468b-9858-cfbcf2518e55") : secret "image-registry-tls" not found Jun 11 21:14:02.635252 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.635105 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2075f225-0880-4abf-807b-378d2d3be472-serving-cert\") pod \"console-operator-5dbbb5d744-tnsxc\" (UID: \"2075f225-0880-4abf-807b-378d2d3be472\") " pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:02.635252 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.635136 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjfzj\" (UniqueName: \"kubernetes.io/projected/2075f225-0880-4abf-807b-378d2d3be472-kube-api-access-xjfzj\") pod \"console-operator-5dbbb5d744-tnsxc\" (UID: \"2075f225-0880-4abf-807b-378d2d3be472\") " pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:02.635252 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.635198 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f6b62f4-15c4-468b-9858-cfbcf2518e55-trusted-ca\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.635252 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.635238 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2075f225-0880-4abf-807b-378d2d3be472-trusted-ca\") pod \"console-operator-5dbbb5d744-tnsxc\" (UID: \"2075f225-0880-4abf-807b-378d2d3be472\") " pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:02.635456 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.635287 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7f6b62f4-15c4-468b-9858-cfbcf2518e55-ca-trust-extracted\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.635456 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.635382 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-default-certificate\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:02.635456 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.635418 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:02.635456 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.635455 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2075f225-0880-4abf-807b-378d2d3be472-config\") pod \"console-operator-5dbbb5d744-tnsxc\" (UID: \"2075f225-0880-4abf-807b-378d2d3be472\") " pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:02.635661 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.635514 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-certificates\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.635661 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:02.635591 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle podName:e4baf798-247a-4237-a103-aa1dae1b1557 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:03.135580528 +0000 UTC m=+216.878660494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle") pod "router-default-5d798db97b-wnl55" (UID: "e4baf798-247a-4237-a103-aa1dae1b1557") : configmap references non-existent config key: service-ca.crt Jun 11 21:14:02.636140 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.636121 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f6b62f4-15c4-468b-9858-cfbcf2518e55-trusted-ca\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.637389 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.637369 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-stats-auth\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:02.637520 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.637500 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7f6b62f4-15c4-468b-9858-cfbcf2518e55-image-registry-private-configuration\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.637582 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.637511 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7f6b62f4-15c4-468b-9858-cfbcf2518e55-installation-pull-secrets\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.637641 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.637589 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-default-certificate\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:02.643188 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.643165 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-bound-sa-token\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.643377 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.643330 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htxsl\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-kube-api-access-htxsl\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:02.643725 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.643701 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj4k4\" (UniqueName: \"kubernetes.io/projected/e4baf798-247a-4237-a103-aa1dae1b1557-kube-api-access-pj4k4\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:02.736707 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.736615 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2075f225-0880-4abf-807b-378d2d3be472-serving-cert\") pod \"console-operator-5dbbb5d744-tnsxc\" (UID: \"2075f225-0880-4abf-807b-378d2d3be472\") " pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:02.736707 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.736650 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjfzj\" (UniqueName: \"kubernetes.io/projected/2075f225-0880-4abf-807b-378d2d3be472-kube-api-access-xjfzj\") pod \"console-operator-5dbbb5d744-tnsxc\" (UID: \"2075f225-0880-4abf-807b-378d2d3be472\") " pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:02.736707 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.736673 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2075f225-0880-4abf-807b-378d2d3be472-trusted-ca\") pod \"console-operator-5dbbb5d744-tnsxc\" (UID: \"2075f225-0880-4abf-807b-378d2d3be472\") " pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:02.736707 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.736700 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2075f225-0880-4abf-807b-378d2d3be472-config\") pod \"console-operator-5dbbb5d744-tnsxc\" (UID: \"2075f225-0880-4abf-807b-378d2d3be472\") " pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:02.737340 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.737314 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2075f225-0880-4abf-807b-378d2d3be472-config\") pod \"console-operator-5dbbb5d744-tnsxc\" (UID: \"2075f225-0880-4abf-807b-378d2d3be472\") " pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:02.737441 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.737421 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2075f225-0880-4abf-807b-378d2d3be472-trusted-ca\") pod \"console-operator-5dbbb5d744-tnsxc\" (UID: \"2075f225-0880-4abf-807b-378d2d3be472\") " pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:02.738957 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.738934 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2075f225-0880-4abf-807b-378d2d3be472-serving-cert\") pod \"console-operator-5dbbb5d744-tnsxc\" (UID: \"2075f225-0880-4abf-807b-378d2d3be472\") " pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:02.744524 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.744503 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjfzj\" (UniqueName: \"kubernetes.io/projected/2075f225-0880-4abf-807b-378d2d3be472-kube-api-access-xjfzj\") pod \"console-operator-5dbbb5d744-tnsxc\" (UID: \"2075f225-0880-4abf-807b-378d2d3be472\") " pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:02.843041 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.842976 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:02.963700 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:02.963668 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5dbbb5d744-tnsxc"] Jun 11 21:14:02.966585 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:14:02.966557 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2075f225_0880_4abf_807b_378d2d3be472.slice/crio-ad1d9139e73fa3056fc29e1cb09d4f13f5ad767954b90e899109f3ee1b4f1ae9 WatchSource:0}: Error finding container ad1d9139e73fa3056fc29e1cb09d4f13f5ad767954b90e899109f3ee1b4f1ae9: Status 404 returned error can't find the container with id ad1d9139e73fa3056fc29e1cb09d4f13f5ad767954b90e899109f3ee1b4f1ae9 Jun 11 21:14:03.140858 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:03.140822 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:03.141068 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:03.140888 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:03.141068 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:03.140930 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:03.141068 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:03.141045 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle podName:e4baf798-247a-4237-a103-aa1dae1b1557 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:04.141009608 +0000 UTC m=+217.884089574 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle") pod "router-default-5d798db97b-wnl55" (UID: "e4baf798-247a-4237-a103-aa1dae1b1557") : configmap references non-existent config key: service-ca.crt Jun 11 21:14:03.141068 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:03.141058 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 21:14:03.141251 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:03.141115 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs podName:e4baf798-247a-4237-a103-aa1dae1b1557 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:04.14109991 +0000 UTC m=+217.884179875 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs") pod "router-default-5d798db97b-wnl55" (UID: "e4baf798-247a-4237-a103-aa1dae1b1557") : secret "router-metrics-certs-default" not found Jun 11 21:14:03.141251 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:03.141060 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 21:14:03.141251 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:03.141133 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7988c46b7d-xgkmt: secret "image-registry-tls" not found Jun 11 21:14:03.141251 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:03.141156 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls podName:7f6b62f4-15c4-468b-9858-cfbcf2518e55 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:04.141149398 +0000 UTC m=+217.884229364 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls") pod "image-registry-7988c46b7d-xgkmt" (UID: "7f6b62f4-15c4-468b-9858-cfbcf2518e55") : secret "image-registry-tls" not found Jun 11 21:14:03.265912 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:03.265881 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" event={"ID":"2075f225-0880-4abf-807b-378d2d3be472","Type":"ContainerStarted","Data":"ad1d9139e73fa3056fc29e1cb09d4f13f5ad767954b90e899109f3ee1b4f1ae9"} Jun 11 21:14:04.147410 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:04.147378 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:04.147889 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:04.147433 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:04.147889 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:04.147494 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:04.147889 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:04.147535 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 21:14:04.147889 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:04.147607 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs podName:e4baf798-247a-4237-a103-aa1dae1b1557 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:06.147586369 +0000 UTC m=+219.890666338 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs") pod "router-default-5d798db97b-wnl55" (UID: "e4baf798-247a-4237-a103-aa1dae1b1557") : secret "router-metrics-certs-default" not found Jun 11 21:14:04.147889 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:04.147617 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 21:14:04.147889 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:04.147638 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7988c46b7d-xgkmt: secret "image-registry-tls" not found Jun 11 21:14:04.147889 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:04.147624 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle podName:e4baf798-247a-4237-a103-aa1dae1b1557 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:06.147616384 +0000 UTC m=+219.890696352 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle") pod "router-default-5d798db97b-wnl55" (UID: "e4baf798-247a-4237-a103-aa1dae1b1557") : configmap references non-existent config key: service-ca.crt Jun 11 21:14:04.147889 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:04.147695 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls podName:7f6b62f4-15c4-468b-9858-cfbcf2518e55 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:06.147677457 +0000 UTC m=+219.890757424 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls") pod "image-registry-7988c46b7d-xgkmt" (UID: "7f6b62f4-15c4-468b-9858-cfbcf2518e55") : secret "image-registry-tls" not found Jun 11 21:14:06.164998 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:06.164945 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:06.165478 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:06.165048 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:06.165478 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:06.165088 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:06.165478 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:06.165132 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle podName:e4baf798-247a-4237-a103-aa1dae1b1557 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:10.165115554 +0000 UTC m=+223.908195525 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle") pod "router-default-5d798db97b-wnl55" (UID: "e4baf798-247a-4237-a103-aa1dae1b1557") : configmap references non-existent config key: service-ca.crt Jun 11 21:14:06.165478 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:06.165188 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 21:14:06.165478 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:06.165240 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs podName:e4baf798-247a-4237-a103-aa1dae1b1557 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:10.165228202 +0000 UTC m=+223.908308167 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs") pod "router-default-5d798db97b-wnl55" (UID: "e4baf798-247a-4237-a103-aa1dae1b1557") : secret "router-metrics-certs-default" not found Jun 11 21:14:06.165478 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:06.165196 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 21:14:06.165478 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:06.165257 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7988c46b7d-xgkmt: secret "image-registry-tls" not found Jun 11 21:14:06.165478 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:06.165288 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls podName:7f6b62f4-15c4-468b-9858-cfbcf2518e55 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:10.165279293 +0000 UTC m=+223.908359260 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls") pod "image-registry-7988c46b7d-xgkmt" (UID: "7f6b62f4-15c4-468b-9858-cfbcf2518e55") : secret "image-registry-tls" not found Jun 11 21:14:06.276460 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:06.276428 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/0.log" Jun 11 21:14:06.276590 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:06.276472 2563 generic.go:356] "Generic (PLEG): container finished" podID="2075f225-0880-4abf-807b-378d2d3be472" containerID="6204cd82518baf6045cc81a9adb2bcd85cbdebe93595b726e9bf48cd8abd07bf" exitCode=255 Jun 11 21:14:06.276590 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:06.276512 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" event={"ID":"2075f225-0880-4abf-807b-378d2d3be472","Type":"ContainerDied","Data":"6204cd82518baf6045cc81a9adb2bcd85cbdebe93595b726e9bf48cd8abd07bf"} Jun 11 21:14:06.276761 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:06.276745 2563 scope.go:117] "RemoveContainer" containerID="6204cd82518baf6045cc81a9adb2bcd85cbdebe93595b726e9bf48cd8abd07bf" Jun 11 21:14:07.280657 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.280632 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/1.log" Jun 11 21:14:07.281055 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.281041 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/0.log" Jun 11 21:14:07.281098 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.281073 2563 generic.go:356] "Generic (PLEG): container finished" podID="2075f225-0880-4abf-807b-378d2d3be472" containerID="193140f4fa951e8b6e6a188432f05d8a92c5f15e7431bfb77674ae74c511984c" exitCode=255 Jun 11 21:14:07.281135 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.281108 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" event={"ID":"2075f225-0880-4abf-807b-378d2d3be472","Type":"ContainerDied","Data":"193140f4fa951e8b6e6a188432f05d8a92c5f15e7431bfb77674ae74c511984c"} Jun 11 21:14:07.281173 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.281146 2563 scope.go:117] "RemoveContainer" containerID="6204cd82518baf6045cc81a9adb2bcd85cbdebe93595b726e9bf48cd8abd07bf" Jun 11 21:14:07.281375 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.281358 2563 scope.go:117] "RemoveContainer" containerID="193140f4fa951e8b6e6a188432f05d8a92c5f15e7431bfb77674ae74c511984c" Jun 11 21:14:07.281559 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:07.281539 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5dbbb5d744-tnsxc_openshift-console-operator(2075f225-0880-4abf-807b-378d2d3be472)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" podUID="2075f225-0880-4abf-807b-378d2d3be472" Jun 11 21:14:07.426068 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.426040 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-x9k48"] Jun 11 21:14:07.430100 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.430081 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:07.432411 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.432386 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 11 21:14:07.432411 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.432386 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 11 21:14:07.432735 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.432717 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-w9gk6\"" Jun 11 21:14:07.432862 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.432764 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 11 21:14:07.432970 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.432956 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 11 21:14:07.440305 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.440282 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x9k48"] Jun 11 21:14:07.477313 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.477285 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-crio-socket\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:07.477448 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.477338 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:07.477448 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.477367 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjjb8\" (UniqueName: \"kubernetes.io/projected/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-api-access-qjjb8\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:07.477448 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.477417 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-data-volume\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:07.477551 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.477459 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:07.578334 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.578283 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjjb8\" (UniqueName: \"kubernetes.io/projected/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-api-access-qjjb8\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:07.578334 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.578342 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-data-volume\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:07.578599 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.578388 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:07.578599 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.578429 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-crio-socket\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:07.578599 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:07.578495 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 21:14:07.578599 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.578528 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:07.578599 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:07.578552 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm podName:3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:08.07853617 +0000 UTC m=+221.821616135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x9k48" (UID: "3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5") : configmap "kube-rbac-proxy" not found Jun 11 21:14:07.578599 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.578498 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-crio-socket\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:07.578899 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:07.578609 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 21:14:07.578899 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:07.578664 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls podName:3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:08.078649325 +0000 UTC m=+221.821729292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x9k48" (UID: "3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5") : secret "insights-runtime-extractor-tls" not found Jun 11 21:14:07.578899 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.578758 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-data-volume\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:07.589797 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:07.589777 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjjb8\" (UniqueName: \"kubernetes.io/projected/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-api-access-qjjb8\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:08.082297 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:08.082260 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:08.082536 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:08.082353 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:08.082536 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:08.082396 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 21:14:08.082536 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:08.082456 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 21:14:08.082536 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:08.082461 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm podName:3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:09.08244676 +0000 UTC m=+222.825526725 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x9k48" (UID: "3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5") : configmap "kube-rbac-proxy" not found Jun 11 21:14:08.082536 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:08.082509 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls podName:3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:09.082494644 +0000 UTC m=+222.825574610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x9k48" (UID: "3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5") : secret "insights-runtime-extractor-tls" not found Jun 11 21:14:08.286584 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:08.286555 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/1.log" Jun 11 21:14:08.286955 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:08.286886 2563 scope.go:117] "RemoveContainer" containerID="193140f4fa951e8b6e6a188432f05d8a92c5f15e7431bfb77674ae74c511984c" Jun 11 21:14:08.287080 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:08.287059 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5dbbb5d744-tnsxc_openshift-console-operator(2075f225-0880-4abf-807b-378d2d3be472)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" podUID="2075f225-0880-4abf-807b-378d2d3be472" Jun 11 21:14:09.092260 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:09.092222 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:09.092423 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:09.092285 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:09.092423 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:09.092362 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 21:14:09.092423 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:09.092378 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 21:14:09.092423 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:09.092413 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm podName:3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:11.092400044 +0000 UTC m=+224.835480010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x9k48" (UID: "3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5") : configmap "kube-rbac-proxy" not found Jun 11 21:14:09.092558 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:09.092431 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls podName:3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:11.092418535 +0000 UTC m=+224.835498502 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x9k48" (UID: "3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5") : secret "insights-runtime-extractor-tls" not found Jun 11 21:14:09.197342 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:09.197314 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9wcfm_a76fe3c1-133a-4c0d-b36f-4f309a013579/dns-node-resolver/0.log" Jun 11 21:14:10.141076 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.141039 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-6txgc"] Jun 11 21:14:10.144088 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.144069 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7b994bc59-6txgc" Jun 11 21:14:10.146522 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.146500 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-v6bf4\"" Jun 11 21:14:10.146649 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.146524 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 11 21:14:10.147455 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.147438 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 11 21:14:10.147512 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.147500 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 11 21:14:10.147695 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.147675 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 11 21:14:10.154538 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.154518 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-6txgc"] Jun 11 21:14:10.202636 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.202601 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cfcf8a28-cfaa-400e-ba3c-0d6f0772880c-signing-key\") pod \"service-ca-7b994bc59-6txgc\" (UID: \"cfcf8a28-cfaa-400e-ba3c-0d6f0772880c\") " pod="openshift-service-ca/service-ca-7b994bc59-6txgc" Jun 11 21:14:10.202803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.202653 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:10.202803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.202683 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:10.202803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.202709 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:10.202803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.202744 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cfcf8a28-cfaa-400e-ba3c-0d6f0772880c-signing-cabundle\") pod \"service-ca-7b994bc59-6txgc\" (UID: \"cfcf8a28-cfaa-400e-ba3c-0d6f0772880c\") " pod="openshift-service-ca/service-ca-7b994bc59-6txgc" Jun 11 21:14:10.202803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.202763 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnhxg\" (UniqueName: \"kubernetes.io/projected/cfcf8a28-cfaa-400e-ba3c-0d6f0772880c-kube-api-access-hnhxg\") pod \"service-ca-7b994bc59-6txgc\" (UID: \"cfcf8a28-cfaa-400e-ba3c-0d6f0772880c\") " pod="openshift-service-ca/service-ca-7b994bc59-6txgc" Jun 11 21:14:10.202803 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:10.202787 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle podName:e4baf798-247a-4237-a103-aa1dae1b1557 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:18.202763956 +0000 UTC m=+231.945843922 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle") pod "router-default-5d798db97b-wnl55" (UID: "e4baf798-247a-4237-a103-aa1dae1b1557") : configmap references non-existent config key: service-ca.crt Jun 11 21:14:10.203054 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:10.202820 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 21:14:10.203054 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:10.202854 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs podName:e4baf798-247a-4237-a103-aa1dae1b1557 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:18.202844825 +0000 UTC m=+231.945924794 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs") pod "router-default-5d798db97b-wnl55" (UID: "e4baf798-247a-4237-a103-aa1dae1b1557") : secret "router-metrics-certs-default" not found Jun 11 21:14:10.203054 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:10.202873 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 21:14:10.203054 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:10.202891 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7988c46b7d-xgkmt: secret "image-registry-tls" not found Jun 11 21:14:10.203054 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:10.202944 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls podName:7f6b62f4-15c4-468b-9858-cfbcf2518e55 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:18.202919157 +0000 UTC m=+231.945999150 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls") pod "image-registry-7988c46b7d-xgkmt" (UID: "7f6b62f4-15c4-468b-9858-cfbcf2518e55") : secret "image-registry-tls" not found Jun 11 21:14:10.303916 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.303879 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cfcf8a28-cfaa-400e-ba3c-0d6f0772880c-signing-cabundle\") pod \"service-ca-7b994bc59-6txgc\" (UID: \"cfcf8a28-cfaa-400e-ba3c-0d6f0772880c\") " pod="openshift-service-ca/service-ca-7b994bc59-6txgc" Jun 11 21:14:10.303916 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.303922 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnhxg\" (UniqueName: \"kubernetes.io/projected/cfcf8a28-cfaa-400e-ba3c-0d6f0772880c-kube-api-access-hnhxg\") pod \"service-ca-7b994bc59-6txgc\" (UID: \"cfcf8a28-cfaa-400e-ba3c-0d6f0772880c\") " pod="openshift-service-ca/service-ca-7b994bc59-6txgc" Jun 11 21:14:10.304222 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.303969 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cfcf8a28-cfaa-400e-ba3c-0d6f0772880c-signing-key\") pod \"service-ca-7b994bc59-6txgc\" (UID: \"cfcf8a28-cfaa-400e-ba3c-0d6f0772880c\") " pod="openshift-service-ca/service-ca-7b994bc59-6txgc" Jun 11 21:14:10.304557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.304536 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cfcf8a28-cfaa-400e-ba3c-0d6f0772880c-signing-cabundle\") pod \"service-ca-7b994bc59-6txgc\" (UID: \"cfcf8a28-cfaa-400e-ba3c-0d6f0772880c\") " pod="openshift-service-ca/service-ca-7b994bc59-6txgc" Jun 11 21:14:10.306476 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.306450 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cfcf8a28-cfaa-400e-ba3c-0d6f0772880c-signing-key\") pod \"service-ca-7b994bc59-6txgc\" (UID: \"cfcf8a28-cfaa-400e-ba3c-0d6f0772880c\") " pod="openshift-service-ca/service-ca-7b994bc59-6txgc" Jun 11 21:14:10.313213 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.313185 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnhxg\" (UniqueName: \"kubernetes.io/projected/cfcf8a28-cfaa-400e-ba3c-0d6f0772880c-kube-api-access-hnhxg\") pod \"service-ca-7b994bc59-6txgc\" (UID: \"cfcf8a28-cfaa-400e-ba3c-0d6f0772880c\") " pod="openshift-service-ca/service-ca-7b994bc59-6txgc" Jun 11 21:14:10.452397 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.452307 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7b994bc59-6txgc" Jun 11 21:14:10.573336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.573305 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-6txgc"] Jun 11 21:14:10.576236 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:14:10.576205 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfcf8a28_cfaa_400e_ba3c_0d6f0772880c.slice/crio-626bf3485bae2f703c66a0dfc3079b62fc8615675d25e0f51f6f63ee9d93cf85 WatchSource:0}: Error finding container 626bf3485bae2f703c66a0dfc3079b62fc8615675d25e0f51f6f63ee9d93cf85: Status 404 returned error can't find the container with id 626bf3485bae2f703c66a0dfc3079b62fc8615675d25e0f51f6f63ee9d93cf85 Jun 11 21:14:10.596000 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:10.595975 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vb7pc_23d74d65-50c4-45b2-bfb0-33bd224279c8/node-ca/0.log" Jun 11 21:14:11.110598 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:11.110556 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:11.110786 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:11.110683 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 21:14:11.110786 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:11.110752 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm podName:3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:15.110729793 +0000 UTC m=+228.853809760 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x9k48" (UID: "3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5") : configmap "kube-rbac-proxy" not found Jun 11 21:14:11.110786 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:11.110687 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:11.110786 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:11.110778 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 21:14:11.111045 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:11.110830 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls podName:3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:15.110814222 +0000 UTC m=+228.853894194 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x9k48" (UID: "3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5") : secret "insights-runtime-extractor-tls" not found Jun 11 21:14:11.297235 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:11.297191 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7b994bc59-6txgc" event={"ID":"cfcf8a28-cfaa-400e-ba3c-0d6f0772880c","Type":"ContainerStarted","Data":"626bf3485bae2f703c66a0dfc3079b62fc8615675d25e0f51f6f63ee9d93cf85"} Jun 11 21:14:11.397754 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:11.397679 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zvmqc_99fddcb2-b296-415e-94a9-15a40dc7462d/serve-healthcheck-canary/0.log" Jun 11 21:14:12.844137 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:12.844104 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:12.844505 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:12.844175 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:12.844570 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:12.844557 2563 scope.go:117] "RemoveContainer" containerID="193140f4fa951e8b6e6a188432f05d8a92c5f15e7431bfb77674ae74c511984c" Jun 11 21:14:12.844759 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:12.844739 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5dbbb5d744-tnsxc_openshift-console-operator(2075f225-0880-4abf-807b-378d2d3be472)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" podUID="2075f225-0880-4abf-807b-378d2d3be472" Jun 11 21:14:13.303887 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:13.303852 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7b994bc59-6txgc" event={"ID":"cfcf8a28-cfaa-400e-ba3c-0d6f0772880c","Type":"ContainerStarted","Data":"22070852853f83dde8d962435e4c80bb5e7623331d043b545bcd0f91e84ee325"} Jun 11 21:14:13.328851 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:13.328765 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7b994bc59-6txgc" podStartSLOduration=1.379367219 podStartE2EDuration="3.328747231s" totalImagesPullingTime="1.949380012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:14:13.327841385 +0000 UTC m=+227.070921374" watchObservedRunningTime="2026-06-11 21:14:13.328747231 +0000 UTC m=+227.071827219" Jun 11 21:14:15.145896 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:15.145842 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:15.146316 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:15.145950 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:15.146316 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:15.146037 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 21:14:15.146316 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:15.146067 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 21:14:15.146316 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:15.146127 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls podName:3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:23.146105804 +0000 UTC m=+236.889185776 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x9k48" (UID: "3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5") : secret "insights-runtime-extractor-tls" not found Jun 11 21:14:15.146316 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:15.146146 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm podName:3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:23.146137289 +0000 UTC m=+236.889217259 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x9k48" (UID: "3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5") : configmap "kube-rbac-proxy" not found Jun 11 21:14:18.273312 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:18.273276 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:18.273312 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:18.273317 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:18.273829 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:18.273335 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:18.273829 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:18.273437 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle podName:e4baf798-247a-4237-a103-aa1dae1b1557 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:34.273420802 +0000 UTC m=+248.016500768 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle") pod "router-default-5d798db97b-wnl55" (UID: "e4baf798-247a-4237-a103-aa1dae1b1557") : configmap references non-existent config key: service-ca.crt Jun 11 21:14:18.273829 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:18.273441 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 21:14:18.273829 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:18.273473 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs podName:e4baf798-247a-4237-a103-aa1dae1b1557 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:34.273466871 +0000 UTC m=+248.016546836 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs") pod "router-default-5d798db97b-wnl55" (UID: "e4baf798-247a-4237-a103-aa1dae1b1557") : secret "router-metrics-certs-default" not found Jun 11 21:14:18.275675 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:18.275659 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls\") pod \"image-registry-7988c46b7d-xgkmt\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:18.342121 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:18.342096 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:18.465904 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:18.465870 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7988c46b7d-xgkmt"] Jun 11 21:14:18.468841 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:14:18.468814 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f6b62f4_15c4_468b_9858_cfbcf2518e55.slice/crio-a364cc3d91512e7e0c5b8015671add69884f099e6d200850b06fde5bd227dea9 WatchSource:0}: Error finding container a364cc3d91512e7e0c5b8015671add69884f099e6d200850b06fde5bd227dea9: Status 404 returned error can't find the container with id a364cc3d91512e7e0c5b8015671add69884f099e6d200850b06fde5bd227dea9 Jun 11 21:14:19.327084 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:19.327053 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" event={"ID":"7f6b62f4-15c4-468b-9858-cfbcf2518e55","Type":"ContainerStarted","Data":"d5a9d5b25e1a2c3db36ac7d59a6e9d6b634eda23b9fd499fc78195cc13c79b61"} Jun 11 21:14:19.327449 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:19.327099 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" event={"ID":"7f6b62f4-15c4-468b-9858-cfbcf2518e55","Type":"ContainerStarted","Data":"a364cc3d91512e7e0c5b8015671add69884f099e6d200850b06fde5bd227dea9"} Jun 11 21:14:19.327449 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:19.327187 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:23.215518 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:23.215472 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:23.215922 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:23.215558 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:23.215922 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:23.215649 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 21:14:23.215922 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:23.215700 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls podName:3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:39.215686587 +0000 UTC m=+252.958766552 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x9k48" (UID: "3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5") : secret "insights-runtime-extractor-tls" not found Jun 11 21:14:23.216220 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:23.216196 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:24.612015 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:24.611983 2563 scope.go:117] "RemoveContainer" containerID="193140f4fa951e8b6e6a188432f05d8a92c5f15e7431bfb77674ae74c511984c" Jun 11 21:14:25.348414 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:25.348387 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:14:25.348783 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:25.348768 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/1.log" Jun 11 21:14:25.348833 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:25.348801 2563 generic.go:356] "Generic (PLEG): container finished" podID="2075f225-0880-4abf-807b-378d2d3be472" containerID="006f5739307374fcf16aa77285d8458e2aff703677754dff370e1bc86b4c319f" exitCode=255 Jun 11 21:14:25.348874 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:25.348837 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" event={"ID":"2075f225-0880-4abf-807b-378d2d3be472","Type":"ContainerDied","Data":"006f5739307374fcf16aa77285d8458e2aff703677754dff370e1bc86b4c319f"} Jun 11 21:14:25.348910 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:25.348880 2563 scope.go:117] "RemoveContainer" containerID="193140f4fa951e8b6e6a188432f05d8a92c5f15e7431bfb77674ae74c511984c" Jun 11 21:14:25.349212 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:25.349190 2563 scope.go:117] "RemoveContainer" containerID="006f5739307374fcf16aa77285d8458e2aff703677754dff370e1bc86b4c319f" Jun 11 21:14:25.349381 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:25.349364 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5dbbb5d744-tnsxc_openshift-console-operator(2075f225-0880-4abf-807b-378d2d3be472)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" podUID="2075f225-0880-4abf-807b-378d2d3be472" Jun 11 21:14:25.364609 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:25.364569 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" podStartSLOduration=23.364556182 podStartE2EDuration="23.364556182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:14:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:14:19.344463971 +0000 UTC m=+233.087543982" watchObservedRunningTime="2026-06-11 21:14:25.364556182 +0000 UTC m=+239.107636170" Jun 11 21:14:26.352594 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:26.352568 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:14:32.843858 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:32.843825 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:32.844266 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:32.843877 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:32.844266 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:32.844168 2563 scope.go:117] "RemoveContainer" containerID="006f5739307374fcf16aa77285d8458e2aff703677754dff370e1bc86b4c319f" Jun 11 21:14:32.844335 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:32.844315 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5dbbb5d744-tnsxc_openshift-console-operator(2075f225-0880-4abf-807b-378d2d3be472)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" podUID="2075f225-0880-4abf-807b-378d2d3be472" Jun 11 21:14:34.305625 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.305578 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:34.305625 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.305637 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:34.306327 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.306307 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4baf798-247a-4237-a103-aa1dae1b1557-service-ca-bundle\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:34.307900 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.307882 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4baf798-247a-4237-a103-aa1dae1b1557-metrics-certs\") pod \"router-default-5d798db97b-wnl55\" (UID: \"e4baf798-247a-4237-a103-aa1dae1b1557\") " pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:34.407337 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.407307 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7988c46b7d-xgkmt"] Jun 11 21:14:34.411480 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.411454 2563 patch_prober.go:28] interesting pod/image-registry-7988c46b7d-xgkmt container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 21:14:34.411621 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.411502 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" podUID="7f6b62f4-15c4-468b-9858-cfbcf2518e55" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 21:14:34.434414 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.434389 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84589dccd5-zrb5r"] Jun 11 21:14:34.438631 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.438616 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.448500 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.448481 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84589dccd5-zrb5r"] Jun 11 21:14:34.548684 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.548656 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-v4wq7\"" Jun 11 21:14:34.557470 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.557399 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:34.608244 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.607981 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-bound-sa-token\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.608244 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.608052 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-ca-trust-extracted\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.608244 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.608103 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-trusted-ca\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.608244 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.608145 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-registry-tls\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.608244 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.608170 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksbpx\" (UniqueName: \"kubernetes.io/projected/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-kube-api-access-ksbpx\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.608244 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.608200 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-installation-pull-secrets\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.609012 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.608254 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-image-registry-private-configuration\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.609012 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.608319 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-registry-certificates\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.685974 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.685943 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d798db97b-wnl55"] Jun 11 21:14:34.688892 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:14:34.688868 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4baf798_247a_4237_a103_aa1dae1b1557.slice/crio-f41f252a35cfe970969246b5c1a8c06c66df4d1b62839e32aa3bba7fa0df00af WatchSource:0}: Error finding container f41f252a35cfe970969246b5c1a8c06c66df4d1b62839e32aa3bba7fa0df00af: Status 404 returned error can't find the container with id f41f252a35cfe970969246b5c1a8c06c66df4d1b62839e32aa3bba7fa0df00af Jun 11 21:14:34.708676 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.708641 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-registry-certificates\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.708676 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.708676 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-bound-sa-token\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.708848 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.708810 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-ca-trust-extracted\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.708908 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.708852 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-trusted-ca\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.708908 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.708902 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-registry-tls\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.709057 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.708926 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksbpx\" (UniqueName: \"kubernetes.io/projected/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-kube-api-access-ksbpx\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.709621 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.709382 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-ca-trust-extracted\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.709621 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.709410 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-installation-pull-secrets\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.709621 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.709478 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-image-registry-private-configuration\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.709621 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.709578 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-registry-certificates\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.709815 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.709787 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-trusted-ca\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.711417 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.711396 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-registry-tls\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.711545 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.711529 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-image-registry-private-configuration\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.711732 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.711714 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-installation-pull-secrets\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.716424 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.716401 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-bound-sa-token\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.716687 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.716672 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksbpx\" (UniqueName: \"kubernetes.io/projected/15b77cd7-0a18-40d8-8c07-2dd97a915b5e-kube-api-access-ksbpx\") pod \"image-registry-84589dccd5-zrb5r\" (UID: \"15b77cd7-0a18-40d8-8c07-2dd97a915b5e\") " pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.747152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.747130 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:34.872506 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:34.872429 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84589dccd5-zrb5r"] Jun 11 21:14:34.876054 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:14:34.876003 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15b77cd7_0a18_40d8_8c07_2dd97a915b5e.slice/crio-8538708833c20182e9ab23d1ab1cc35f575f97bdaa726a8dee1e0042fc4bd7c9 WatchSource:0}: Error finding container 8538708833c20182e9ab23d1ab1cc35f575f97bdaa726a8dee1e0042fc4bd7c9: Status 404 returned error can't find the container with id 8538708833c20182e9ab23d1ab1cc35f575f97bdaa726a8dee1e0042fc4bd7c9 Jun 11 21:14:35.379978 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:35.379939 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d798db97b-wnl55" event={"ID":"e4baf798-247a-4237-a103-aa1dae1b1557","Type":"ContainerStarted","Data":"1461d3e39e780ac5cc50f5c858aeba0d2f33d4c3fbd825fd505ce7a8c88aa816"} Jun 11 21:14:35.380419 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:35.379992 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d798db97b-wnl55" event={"ID":"e4baf798-247a-4237-a103-aa1dae1b1557","Type":"ContainerStarted","Data":"f41f252a35cfe970969246b5c1a8c06c66df4d1b62839e32aa3bba7fa0df00af"} Jun 11 21:14:35.381368 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:35.381347 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" event={"ID":"15b77cd7-0a18-40d8-8c07-2dd97a915b5e","Type":"ContainerStarted","Data":"88077ea501255de47db0c5ba37aa941ee2e001490d000818d1dc3078459359a0"} Jun 11 21:14:35.381467 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:35.381380 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" event={"ID":"15b77cd7-0a18-40d8-8c07-2dd97a915b5e","Type":"ContainerStarted","Data":"8538708833c20182e9ab23d1ab1cc35f575f97bdaa726a8dee1e0042fc4bd7c9"} Jun 11 21:14:35.381505 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:35.381468 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:14:35.398204 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:35.398154 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5d798db97b-wnl55" podStartSLOduration=33.398136642 podStartE2EDuration="33.398136642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:14:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:14:35.397707158 +0000 UTC m=+249.140787146" watchObservedRunningTime="2026-06-11 21:14:35.398136642 +0000 UTC m=+249.141216632" Jun 11 21:14:35.415178 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:35.415134 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" podStartSLOduration=1.415119989 podStartE2EDuration="1.415119989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:14:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:14:35.413900117 +0000 UTC m=+249.156980108" watchObservedRunningTime="2026-06-11 21:14:35.415119989 +0000 UTC m=+249.158199977" Jun 11 21:14:35.558279 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:35.558248 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:35.561153 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:35.561128 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:36.384674 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:36.384643 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:36.385877 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:36.385858 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5d798db97b-wnl55" Jun 11 21:14:38.540533 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:38.540489 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs\") pod \"network-metrics-daemon-hz8r8\" (UID: \"db4e8904-7461-4943-909f-6e9fe29dea61\") " pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:14:38.542940 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:38.542905 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e8904-7461-4943-909f-6e9fe29dea61-metrics-certs\") pod \"network-metrics-daemon-hz8r8\" (UID: \"db4e8904-7461-4943-909f-6e9fe29dea61\") " pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:14:38.815212 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:38.815167 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-thzvh\"" Jun 11 21:14:38.823097 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:38.823074 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hz8r8" Jun 11 21:14:38.956601 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:38.956565 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hz8r8"] Jun 11 21:14:38.962667 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:14:38.962639 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb4e8904_7461_4943_909f_6e9fe29dea61.slice/crio-6baeb847fa231eca1a6086a08d59ddda9deba9b2d5047d0376ed735092d577f3 WatchSource:0}: Error finding container 6baeb847fa231eca1a6086a08d59ddda9deba9b2d5047d0376ed735092d577f3: Status 404 returned error can't find the container with id 6baeb847fa231eca1a6086a08d59ddda9deba9b2d5047d0376ed735092d577f3 Jun 11 21:14:39.245643 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:39.245549 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:39.247952 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:39.247917 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x9k48\" (UID: \"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5\") " pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:39.395357 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:39.395317 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hz8r8" event={"ID":"db4e8904-7461-4943-909f-6e9fe29dea61","Type":"ContainerStarted","Data":"6baeb847fa231eca1a6086a08d59ddda9deba9b2d5047d0376ed735092d577f3"} Jun 11 21:14:39.540863 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:39.540778 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-w9gk6\"" Jun 11 21:14:39.548882 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:39.548847 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x9k48" Jun 11 21:14:39.686654 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:39.686583 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x9k48"] Jun 11 21:14:39.874146 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:14:39.874116 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f13317f_8f3c_41c5_bd66_3b0cccb0a1e5.slice/crio-895889db0d2a7f7bd8ff51ce664b7e545eb409f4153aff3e763e26f23ea9a5eb WatchSource:0}: Error finding container 895889db0d2a7f7bd8ff51ce664b7e545eb409f4153aff3e763e26f23ea9a5eb: Status 404 returned error can't find the container with id 895889db0d2a7f7bd8ff51ce664b7e545eb409f4153aff3e763e26f23ea9a5eb Jun 11 21:14:40.404048 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:40.403711 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hz8r8" event={"ID":"db4e8904-7461-4943-909f-6e9fe29dea61","Type":"ContainerStarted","Data":"49dc46412f5fa28c7d5c5b3a7d1956e89a90c2530c18657f27f7ff9c3ab66c7a"} Jun 11 21:14:40.404048 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:40.403775 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hz8r8" event={"ID":"db4e8904-7461-4943-909f-6e9fe29dea61","Type":"ContainerStarted","Data":"ce3f6805b8ac4bf3cb275f1b5a4b89b0f732add5d6748509efc8f69ab708421f"} Jun 11 21:14:40.405705 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:40.405641 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x9k48" event={"ID":"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5","Type":"ContainerStarted","Data":"895889db0d2a7f7bd8ff51ce664b7e545eb409f4153aff3e763e26f23ea9a5eb"} Jun 11 21:14:40.421654 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:40.421560 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hz8r8" podStartSLOduration=253.467338958 podStartE2EDuration="4m14.421540355s" totalImagesPullingTime="954.201397ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:14:40.420516676 +0000 UTC m=+254.163596669" watchObservedRunningTime="2026-06-11 21:14:40.421540355 +0000 UTC m=+254.164620342" Jun 11 21:14:43.417834 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:43.417793 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x9k48" event={"ID":"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5","Type":"ContainerStarted","Data":"6bbf79406fe743b2325e58e25f399b26bd0998841f101bf45880d89bc32d1140"} Jun 11 21:14:44.413574 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:44.413533 2563 patch_prober.go:28] interesting pod/image-registry-7988c46b7d-xgkmt container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 21:14:44.413768 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:44.413594 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" podUID="7f6b62f4-15c4-468b-9858-cfbcf2518e55" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 21:14:44.423776 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:44.423741 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x9k48" event={"ID":"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5","Type":"ContainerStarted","Data":"650f049610080532708dce7501de9a5f7c3aa8213277fca3f36982290c9e9f4a"} Jun 11 21:14:46.432288 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:46.432255 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x9k48" event={"ID":"3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5","Type":"ContainerStarted","Data":"abe492dc0101541c2385f8ce34bfc2b9e054f8816b5eb96812f981ac63a9cd31"} Jun 11 21:14:46.449666 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:46.449617 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-x9k48" podStartSLOduration=33.964085688 podStartE2EDuration="39.449604509s" totalImagesPullingTime="5.485518821s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:14:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:14:46.448186502 +0000 UTC m=+260.191266490" watchObservedRunningTime="2026-06-11 21:14:46.449604509 +0000 UTC m=+260.192684494" Jun 11 21:14:46.969644 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:46.969606 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-67vf2"] Jun 11 21:14:46.974234 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:46.974202 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:46.976466 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:46.976442 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 11 21:14:46.976607 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:46.976543 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 11 21:14:46.976607 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:46.976593 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 11 21:14:46.976730 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:46.976634 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 11 21:14:46.976812 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:46.976794 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zl2ql\"" Jun 11 21:14:46.977170 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:46.977152 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 11 21:14:46.977279 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:46.977191 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 11 21:14:47.010301 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.010263 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk9gg\" (UniqueName: \"kubernetes.io/projected/e41a1472-8802-45ed-a95d-88814c60cdd1-kube-api-access-dk9gg\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.010474 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.010311 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e41a1472-8802-45ed-a95d-88814c60cdd1-sys\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.010474 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.010389 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-wtmp\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.010571 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.010472 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e41a1472-8802-45ed-a95d-88814c60cdd1-root\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.010571 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.010520 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-accelerators-collector-config\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.010671 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.010576 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-tls\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.010671 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.010632 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-textfile\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.010769 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.010669 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e41a1472-8802-45ed-a95d-88814c60cdd1-metrics-client-ca\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.010769 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.010696 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.111198 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.111154 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e41a1472-8802-45ed-a95d-88814c60cdd1-sys\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.111198 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.111202 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-wtmp\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.111463 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.111220 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e41a1472-8802-45ed-a95d-88814c60cdd1-root\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.111463 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.111241 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-accelerators-collector-config\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.111463 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.111285 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-tls\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.111463 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.111285 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e41a1472-8802-45ed-a95d-88814c60cdd1-sys\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.111463 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.111302 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e41a1472-8802-45ed-a95d-88814c60cdd1-root\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.111463 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.111306 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-textfile\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.111463 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.111350 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e41a1472-8802-45ed-a95d-88814c60cdd1-metrics-client-ca\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.111463 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.111373 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.111463 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.111375 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-wtmp\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.111463 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:47.111405 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 11 21:14:47.111463 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.111411 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk9gg\" (UniqueName: \"kubernetes.io/projected/e41a1472-8802-45ed-a95d-88814c60cdd1-kube-api-access-dk9gg\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.111463 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:14:47.111463 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-tls podName:e41a1472-8802-45ed-a95d-88814c60cdd1 nodeName:}" failed. No retries permitted until 2026-06-11 21:14:47.61144396 +0000 UTC m=+261.354523931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-tls") pod "node-exporter-67vf2" (UID: "e41a1472-8802-45ed-a95d-88814c60cdd1") : secret "node-exporter-tls" not found Jun 11 21:14:47.112059 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.111683 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-textfile\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.112059 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.111965 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-accelerators-collector-config\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.112140 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.112080 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e41a1472-8802-45ed-a95d-88814c60cdd1-metrics-client-ca\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.113706 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.113688 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.121138 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.121113 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk9gg\" (UniqueName: \"kubernetes.io/projected/e41a1472-8802-45ed-a95d-88814c60cdd1-kube-api-access-dk9gg\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.611183 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.611155 2563 scope.go:117] "RemoveContainer" containerID="006f5739307374fcf16aa77285d8458e2aff703677754dff370e1bc86b4c319f" Jun 11 21:14:47.615651 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.615634 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-tls\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.617880 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.617859 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e41a1472-8802-45ed-a95d-88814c60cdd1-node-exporter-tls\") pod \"node-exporter-67vf2\" (UID: \"e41a1472-8802-45ed-a95d-88814c60cdd1\") " pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.886142 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:47.886062 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-67vf2" Jun 11 21:14:47.899932 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:14:47.899906 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode41a1472_8802_45ed_a95d_88814c60cdd1.slice/crio-06ff995e1417b487feeb76e37ecf50d3297d80d5741c42c7b4f72ddf084dc79e WatchSource:0}: Error finding container 06ff995e1417b487feeb76e37ecf50d3297d80d5741c42c7b4f72ddf084dc79e: Status 404 returned error can't find the container with id 06ff995e1417b487feeb76e37ecf50d3297d80d5741c42c7b4f72ddf084dc79e Jun 11 21:14:48.440567 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:48.440531 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-67vf2" event={"ID":"e41a1472-8802-45ed-a95d-88814c60cdd1","Type":"ContainerStarted","Data":"06ff995e1417b487feeb76e37ecf50d3297d80d5741c42c7b4f72ddf084dc79e"} Jun 11 21:14:48.442522 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:48.442496 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:14:48.442654 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:48.442556 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" event={"ID":"2075f225-0880-4abf-807b-378d2d3be472","Type":"ContainerStarted","Data":"2c997c7e2e55096c88fc24f4cdac9656e1231a16397076131e6835cdb652ca3d"} Jun 11 21:14:48.442887 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:48.442838 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:48.447289 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:48.447266 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" Jun 11 21:14:48.458428 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:48.458372 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5dbbb5d744-tnsxc" podStartSLOduration=44.033803349 podStartE2EDuration="46.458355611s" totalImagesPullingTime="2.424552262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:14:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:14:48.457727885 +0000 UTC m=+262.200807875" watchObservedRunningTime="2026-06-11 21:14:48.458355611 +0000 UTC m=+262.201435600" Jun 11 21:14:49.447366 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:49.447326 2563 generic.go:356] "Generic (PLEG): container finished" podID="e41a1472-8802-45ed-a95d-88814c60cdd1" containerID="e55bd5cc8196ff95c302e5504e136770789bb89e32762850542efce1dcd04ec3" exitCode=0 Jun 11 21:14:49.447744 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:49.447409 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-67vf2" event={"ID":"e41a1472-8802-45ed-a95d-88814c60cdd1","Type":"ContainerDied","Data":"e55bd5cc8196ff95c302e5504e136770789bb89e32762850542efce1dcd04ec3"} Jun 11 21:14:50.453358 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:50.453322 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-67vf2" event={"ID":"e41a1472-8802-45ed-a95d-88814c60cdd1","Type":"ContainerStarted","Data":"969968553c23ee6626d82c85e9a8c8e38dd5570dba57be4cd743b5e3cb85fb44"} Jun 11 21:14:50.453846 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:50.453388 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-67vf2" event={"ID":"e41a1472-8802-45ed-a95d-88814c60cdd1","Type":"ContainerStarted","Data":"91572b5212d9fe298296bd956bbf1631e8b4f3ea1989d12028ae547710cceda0"} Jun 11 21:14:50.471201 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:50.471149 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-67vf2" podStartSLOduration=3.353179909 podStartE2EDuration="4.471132129s" totalImagesPullingTime="1.05262818s" totalInitContainerRuntime="65.32404ms" isStatefulPod=true podCreationTimestamp="2026-06-11 21:14:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:14:50.47032085 +0000 UTC m=+264.213400840" watchObservedRunningTime="2026-06-11 21:14:50.471132129 +0000 UTC m=+264.214212116" Jun 11 21:14:51.256192 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.256159 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-566f7cc5bb-p4wjz"] Jun 11 21:14:51.259331 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.259316 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.262659 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.262630 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 11 21:14:51.262659 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.262648 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 11 21:14:51.262659 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.262659 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 11 21:14:51.262923 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.262662 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bng0svghsjkbc\"" Jun 11 21:14:51.262923 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.262690 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-fxpfh\"" Jun 11 21:14:51.262923 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.262749 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 11 21:14:51.268677 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.268656 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-566f7cc5bb-p4wjz"] Jun 11 21:14:51.347469 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.347431 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.347680 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.347485 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9dlv\" (UniqueName: \"kubernetes.io/projected/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-kube-api-access-f9dlv\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.347680 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.347524 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-secret-metrics-server-client-certs\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.347680 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.347588 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-audit-log\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.347680 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.347620 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-client-ca-bundle\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.347680 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.347647 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-secret-metrics-server-tls\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.347872 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.347714 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-metrics-server-audit-profiles\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.448196 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.448161 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9dlv\" (UniqueName: \"kubernetes.io/projected/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-kube-api-access-f9dlv\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.448400 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.448218 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-secret-metrics-server-client-certs\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.448469 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.448408 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-audit-log\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.448469 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.448456 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-client-ca-bundle\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.448595 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.448480 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-secret-metrics-server-tls\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.448595 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.448505 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-metrics-server-audit-profiles\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.448595 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.448546 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.448851 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.448827 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-audit-log\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.449358 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.449333 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.449462 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.449389 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-metrics-server-audit-profiles\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.450963 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.450931 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-secret-metrics-server-client-certs\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.451064 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.450989 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-secret-metrics-server-tls\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.451109 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.451067 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-client-ca-bundle\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.455854 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.455833 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9dlv\" (UniqueName: \"kubernetes.io/projected/d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2-kube-api-access-f9dlv\") pod \"metrics-server-566f7cc5bb-p4wjz\" (UID: \"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2\") " pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.569009 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.568982 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:14:51.697372 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:51.697315 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-566f7cc5bb-p4wjz"] Jun 11 21:14:51.702732 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:14:51.702655 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd42f6c3d_971b_4ebd_b47a_5fed2c8e07f2.slice/crio-e80e666a7eb8e65eeb1b729f772a9a4ee1469aa7819953873c0d4d6a3d074f29 WatchSource:0}: Error finding container e80e666a7eb8e65eeb1b729f772a9a4ee1469aa7819953873c0d4d6a3d074f29: Status 404 returned error can't find the container with id e80e666a7eb8e65eeb1b729f772a9a4ee1469aa7819953873c0d4d6a3d074f29 Jun 11 21:14:52.462205 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:52.462171 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" event={"ID":"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2","Type":"ContainerStarted","Data":"e80e666a7eb8e65eeb1b729f772a9a4ee1469aa7819953873c0d4d6a3d074f29"} Jun 11 21:14:53.163149 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.163116 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 21:14:53.166977 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.166952 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.169269 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.169243 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 11 21:14:53.169424 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.169308 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 11 21:14:53.169424 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.169321 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 11 21:14:53.169424 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.169314 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-nglp3a1a03ua\"" Jun 11 21:14:53.169811 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.169722 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 11 21:14:53.169811 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.169734 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 11 21:14:53.169811 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.169727 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 11 21:14:53.171198 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.170908 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 11 21:14:53.171198 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.170923 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 11 21:14:53.171198 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.170963 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 11 21:14:53.171198 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.171036 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 11 21:14:53.171198 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.171064 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 11 21:14:53.171198 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.171158 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 11 21:14:53.171547 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.171442 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 11 21:14:53.171752 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.171732 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-svl29\"" Jun 11 21:14:53.172604 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.172587 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 11 21:14:53.182353 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.182333 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 21:14:53.263103 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263071 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263268 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263146 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263268 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263200 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263389 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263272 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj5cm\" (UniqueName: \"kubernetes.io/projected/a727a830-51f9-4fb8-bbd3-498269c1f3ac-kube-api-access-nj5cm\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263389 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263364 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-web-config\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263490 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263400 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263490 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263439 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263591 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263515 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263591 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263559 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a727a830-51f9-4fb8-bbd3-498269c1f3ac-config-out\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263591 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263582 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263738 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263608 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263738 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263638 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263738 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263722 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263881 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263757 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263881 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263790 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263881 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263839 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.263881 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263857 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-config\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.264065 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263880 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.264065 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263915 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a727a830-51f9-4fb8-bbd3-498269c1f3ac-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.264065 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.263945 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.364739 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.364715 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a727a830-51f9-4fb8-bbd3-498269c1f3ac-config-out\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.364850 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.364746 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.364850 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.364764 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.364850 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.364781 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.364850 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.364811 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.364850 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.364834 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.365118 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.364863 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.365118 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.364904 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.365118 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.364928 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-config\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.365118 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.364950 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.365118 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.364976 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a727a830-51f9-4fb8-bbd3-498269c1f3ac-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.365118 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.364999 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.365118 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.365041 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.365118 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.365070 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.365118 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.365094 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.365118 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.365119 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj5cm\" (UniqueName: \"kubernetes.io/projected/a727a830-51f9-4fb8-bbd3-498269c1f3ac-kube-api-access-nj5cm\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.365603 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.365157 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-web-config\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.365603 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.365188 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.365603 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.365218 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.365603 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.365228 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.365603 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.365292 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.367763 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.365942 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.367763 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.366755 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.367763 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.366781 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.367763 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.366932 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.367763 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.367459 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.367763 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.367587 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.368394 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.368367 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.368720 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.368693 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a727a830-51f9-4fb8-bbd3-498269c1f3ac-config-out\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.368844 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.368821 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.369690 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.369666 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.369982 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.369869 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.370199 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.370180 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-web-config\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.370348 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.370255 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-config\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.371104 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.371079 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a727a830-51f9-4fb8-bbd3-498269c1f3ac-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.371185 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.371166 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.371520 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.371501 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.371613 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.371589 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.371648 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.371609 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.376481 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.376462 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj5cm\" (UniqueName: \"kubernetes.io/projected/a727a830-51f9-4fb8-bbd3-498269c1f3ac-kube-api-access-nj5cm\") pod \"prometheus-k8s-0\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.467050 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.466953 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" event={"ID":"d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2","Type":"ContainerStarted","Data":"ca1b89ba4677500423d9d8ef4a604271451a71fb04e0efb80d8e411825bc9fa5"} Jun 11 21:14:53.478792 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.478769 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:14:53.482697 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.482658 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" podStartSLOduration=0.867061781 podStartE2EDuration="2.482644958s" totalImagesPullingTime="1.615583177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:14:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:14:53.481384449 +0000 UTC m=+267.224464440" watchObservedRunningTime="2026-06-11 21:14:53.482644958 +0000 UTC m=+267.225724946" Jun 11 21:14:53.610249 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:53.610215 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 21:14:53.613418 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:14:53.613392 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda727a830_51f9_4fb8_bbd3_498269c1f3ac.slice/crio-5bf833021c54ec193080ff2539f440b16544cf38c85a1fee65a3be400ff64dee WatchSource:0}: Error finding container 5bf833021c54ec193080ff2539f440b16544cf38c85a1fee65a3be400ff64dee: Status 404 returned error can't find the container with id 5bf833021c54ec193080ff2539f440b16544cf38c85a1fee65a3be400ff64dee Jun 11 21:14:54.412198 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:54.412159 2563 patch_prober.go:28] interesting pod/image-registry-7988c46b7d-xgkmt container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 21:14:54.412397 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:54.412221 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" podUID="7f6b62f4-15c4-468b-9858-cfbcf2518e55" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 21:14:54.471661 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:54.471619 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a727a830-51f9-4fb8-bbd3-498269c1f3ac","Type":"ContainerStarted","Data":"5bf833021c54ec193080ff2539f440b16544cf38c85a1fee65a3be400ff64dee"} Jun 11 21:14:54.751343 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:54.751275 2563 patch_prober.go:28] interesting pod/image-registry-84589dccd5-zrb5r container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 21:14:54.751343 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:54.751318 2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" podUID="15b77cd7-0a18-40d8-8c07-2dd97a915b5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 21:14:55.478196 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:55.477653 2563 generic.go:356] "Generic (PLEG): container finished" podID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerID="cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71" exitCode=0 Jun 11 21:14:55.478196 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:55.477755 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a727a830-51f9-4fb8-bbd3-498269c1f3ac","Type":"ContainerDied","Data":"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71"} Jun 11 21:14:56.389121 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:56.389079 2563 patch_prober.go:28] interesting pod/image-registry-84589dccd5-zrb5r container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 21:14:56.389285 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:56.389135 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" podUID="15b77cd7-0a18-40d8-8c07-2dd97a915b5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 21:14:58.493223 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:58.493193 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a727a830-51f9-4fb8-bbd3-498269c1f3ac","Type":"ContainerStarted","Data":"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe"} Jun 11 21:14:59.426697 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.426629 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" podUID="7f6b62f4-15c4-468b-9858-cfbcf2518e55" containerName="registry" containerID="cri-o://d5a9d5b25e1a2c3db36ac7d59a6e9d6b634eda23b9fd499fc78195cc13c79b61" gracePeriod=30 Jun 11 21:14:59.498229 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.498198 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a727a830-51f9-4fb8-bbd3-498269c1f3ac","Type":"ContainerStarted","Data":"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561"} Jun 11 21:14:59.676608 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.676586 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:14:59.829563 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.829527 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7f6b62f4-15c4-468b-9858-cfbcf2518e55-installation-pull-secrets\") pod \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " Jun 11 21:14:59.829738 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.829578 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htxsl\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-kube-api-access-htxsl\") pod \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " Jun 11 21:14:59.829738 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.829611 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-bound-sa-token\") pod \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " Jun 11 21:14:59.829872 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.829741 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls\") pod \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " Jun 11 21:14:59.829872 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.829788 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-certificates\") pod \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " Jun 11 21:14:59.829872 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.829833 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f6b62f4-15c4-468b-9858-cfbcf2518e55-trusted-ca\") pod \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " Jun 11 21:14:59.830055 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.829911 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7f6b62f4-15c4-468b-9858-cfbcf2518e55-image-registry-private-configuration\") pod \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " Jun 11 21:14:59.830055 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.829955 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7f6b62f4-15c4-468b-9858-cfbcf2518e55-ca-trust-extracted\") pod \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\" (UID: \"7f6b62f4-15c4-468b-9858-cfbcf2518e55\") " Jun 11 21:14:59.830542 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.830353 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f6b62f4-15c4-468b-9858-cfbcf2518e55-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7f6b62f4-15c4-468b-9858-cfbcf2518e55" (UID: "7f6b62f4-15c4-468b-9858-cfbcf2518e55"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:14:59.830654 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.830626 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7f6b62f4-15c4-468b-9858-cfbcf2518e55" (UID: "7f6b62f4-15c4-468b-9858-cfbcf2518e55"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:14:59.832611 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.832575 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7f6b62f4-15c4-468b-9858-cfbcf2518e55" (UID: "7f6b62f4-15c4-468b-9858-cfbcf2518e55"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:14:59.832728 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.832580 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f6b62f4-15c4-468b-9858-cfbcf2518e55-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7f6b62f4-15c4-468b-9858-cfbcf2518e55" (UID: "7f6b62f4-15c4-468b-9858-cfbcf2518e55"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:14:59.832810 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.832783 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f6b62f4-15c4-468b-9858-cfbcf2518e55-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7f6b62f4-15c4-468b-9858-cfbcf2518e55" (UID: "7f6b62f4-15c4-468b-9858-cfbcf2518e55"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:14:59.832866 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.832838 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-kube-api-access-htxsl" (OuterVolumeSpecName: "kube-api-access-htxsl") pod "7f6b62f4-15c4-468b-9858-cfbcf2518e55" (UID: "7f6b62f4-15c4-468b-9858-cfbcf2518e55"). InnerVolumeSpecName "kube-api-access-htxsl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:14:59.833251 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.833232 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7f6b62f4-15c4-468b-9858-cfbcf2518e55" (UID: "7f6b62f4-15c4-468b-9858-cfbcf2518e55"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:14:59.839430 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.839404 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f6b62f4-15c4-468b-9858-cfbcf2518e55-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7f6b62f4-15c4-468b-9858-cfbcf2518e55" (UID: "7f6b62f4-15c4-468b-9858-cfbcf2518e55"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 21:14:59.931374 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.931342 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f6b62f4-15c4-468b-9858-cfbcf2518e55-trusted-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:14:59.931374 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.931371 2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7f6b62f4-15c4-468b-9858-cfbcf2518e55-image-registry-private-configuration\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:14:59.931374 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.931381 2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7f6b62f4-15c4-468b-9858-cfbcf2518e55-ca-trust-extracted\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:14:59.931597 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.931390 2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7f6b62f4-15c4-468b-9858-cfbcf2518e55-installation-pull-secrets\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:14:59.931597 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.931398 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htxsl\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-kube-api-access-htxsl\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:14:59.931597 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.931407 2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-bound-sa-token\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:14:59.931597 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.931415 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:14:59.931597 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:14:59.931424 2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7f6b62f4-15c4-468b-9858-cfbcf2518e55-registry-certificates\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:15:00.501982 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:00.501955 2563 generic.go:356] "Generic (PLEG): container finished" podID="7f6b62f4-15c4-468b-9858-cfbcf2518e55" containerID="d5a9d5b25e1a2c3db36ac7d59a6e9d6b634eda23b9fd499fc78195cc13c79b61" exitCode=0 Jun 11 21:15:00.502315 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:00.502002 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" Jun 11 21:15:00.502315 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:00.502005 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" event={"ID":"7f6b62f4-15c4-468b-9858-cfbcf2518e55","Type":"ContainerDied","Data":"d5a9d5b25e1a2c3db36ac7d59a6e9d6b634eda23b9fd499fc78195cc13c79b61"} Jun 11 21:15:00.502315 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:00.502073 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7988c46b7d-xgkmt" event={"ID":"7f6b62f4-15c4-468b-9858-cfbcf2518e55","Type":"ContainerDied","Data":"a364cc3d91512e7e0c5b8015671add69884f099e6d200850b06fde5bd227dea9"} Jun 11 21:15:00.502315 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:00.502102 2563 scope.go:117] "RemoveContainer" containerID="d5a9d5b25e1a2c3db36ac7d59a6e9d6b634eda23b9fd499fc78195cc13c79b61" Jun 11 21:15:00.516816 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:00.516797 2563 scope.go:117] "RemoveContainer" containerID="d5a9d5b25e1a2c3db36ac7d59a6e9d6b634eda23b9fd499fc78195cc13c79b61" Jun 11 21:15:00.517151 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:15:00.517131 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5a9d5b25e1a2c3db36ac7d59a6e9d6b634eda23b9fd499fc78195cc13c79b61\": container with ID starting with d5a9d5b25e1a2c3db36ac7d59a6e9d6b634eda23b9fd499fc78195cc13c79b61 not found: ID does not exist" containerID="d5a9d5b25e1a2c3db36ac7d59a6e9d6b634eda23b9fd499fc78195cc13c79b61" Jun 11 21:15:00.517246 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:00.517159 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a9d5b25e1a2c3db36ac7d59a6e9d6b634eda23b9fd499fc78195cc13c79b61"} err="failed to get container status \"d5a9d5b25e1a2c3db36ac7d59a6e9d6b634eda23b9fd499fc78195cc13c79b61\": rpc error: code = NotFound desc = could not find container \"d5a9d5b25e1a2c3db36ac7d59a6e9d6b634eda23b9fd499fc78195cc13c79b61\": container with ID starting with d5a9d5b25e1a2c3db36ac7d59a6e9d6b634eda23b9fd499fc78195cc13c79b61 not found: ID does not exist" Jun 11 21:15:00.527892 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:00.527871 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7988c46b7d-xgkmt"] Jun 11 21:15:00.531937 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:00.531884 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7988c46b7d-xgkmt"] Jun 11 21:15:00.617222 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:00.616849 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f6b62f4-15c4-468b-9858-cfbcf2518e55" path="/var/lib/kubelet/pods/7f6b62f4-15c4-468b-9858-cfbcf2518e55/volumes" Jun 11 21:15:01.508550 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:01.508507 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a727a830-51f9-4fb8-bbd3-498269c1f3ac","Type":"ContainerStarted","Data":"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d"} Jun 11 21:15:01.508987 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:01.508614 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a727a830-51f9-4fb8-bbd3-498269c1f3ac","Type":"ContainerStarted","Data":"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c"} Jun 11 21:15:01.508987 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:01.508663 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a727a830-51f9-4fb8-bbd3-498269c1f3ac","Type":"ContainerStarted","Data":"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9"} Jun 11 21:15:01.508987 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:01.508692 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a727a830-51f9-4fb8-bbd3-498269c1f3ac","Type":"ContainerStarted","Data":"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b"} Jun 11 21:15:01.537049 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:01.536980 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.767030111 podStartE2EDuration="8.536962958s" totalImagesPullingTime="5.75168039s" totalInitContainerRuntime="18.252457ms" isStatefulPod=false podCreationTimestamp="2026-06-11 21:14:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:15:01.535151332 +0000 UTC m=+275.278231339" watchObservedRunningTime="2026-06-11 21:15:01.536962958 +0000 UTC m=+275.280042948" Jun 11 21:15:03.479645 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:03.479613 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:15:04.751345 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:04.751310 2563 patch_prober.go:28] interesting pod/image-registry-84589dccd5-zrb5r container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 21:15:04.751740 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:04.751359 2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" podUID="15b77cd7-0a18-40d8-8c07-2dd97a915b5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 21:15:06.389590 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:06.389558 2563 patch_prober.go:28] interesting pod/image-registry-84589dccd5-zrb5r container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 21:15:06.389967 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:06.389610 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" podUID="15b77cd7-0a18-40d8-8c07-2dd97a915b5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 21:15:08.123822 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:15:08.123760 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5qxs8" podUID="faf3cd10-7238-4409-bf88-932cd0b1c14e" Jun 11 21:15:08.532765 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:08.532689 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5qxs8" Jun 11 21:15:11.025851 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:11.025811 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:15:11.028140 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:11.028118 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/faf3cd10-7238-4409-bf88-932cd0b1c14e-metrics-tls\") pod \"dns-default-5qxs8\" (UID: \"faf3cd10-7238-4409-bf88-932cd0b1c14e\") " pod="openshift-dns/dns-default-5qxs8" Jun 11 21:15:11.236632 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:11.236601 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6v9mc\"" Jun 11 21:15:11.244489 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:11.244470 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5qxs8" Jun 11 21:15:11.377169 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:11.377139 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5qxs8"] Jun 11 21:15:11.380280 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:15:11.380249 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf3cd10_7238_4409_bf88_932cd0b1c14e.slice/crio-d75e352b499891ad892cbddd014ad42e04f1ee9423e055b8fdb435abb355c83e WatchSource:0}: Error finding container d75e352b499891ad892cbddd014ad42e04f1ee9423e055b8fdb435abb355c83e: Status 404 returned error can't find the container with id d75e352b499891ad892cbddd014ad42e04f1ee9423e055b8fdb435abb355c83e Jun 11 21:15:11.548170 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:11.548133 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5qxs8" event={"ID":"faf3cd10-7238-4409-bf88-932cd0b1c14e","Type":"ContainerStarted","Data":"d75e352b499891ad892cbddd014ad42e04f1ee9423e055b8fdb435abb355c83e"} Jun 11 21:15:11.569504 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:11.569474 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:15:11.569647 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:11.569525 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:15:13.557311 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:13.557273 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5qxs8" event={"ID":"faf3cd10-7238-4409-bf88-932cd0b1c14e","Type":"ContainerStarted","Data":"2710475eaf2da2912729f706d96acaecd2dcf022e6542eef05016ddf7e72ab46"} Jun 11 21:15:13.557722 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:13.557340 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5qxs8" event={"ID":"faf3cd10-7238-4409-bf88-932cd0b1c14e","Type":"ContainerStarted","Data":"020924f36bf29caccdc58ead042d77e235811bb3b4de868174bcffd3d2473aa4"} Jun 11 21:15:13.557722 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:13.557386 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5qxs8" Jun 11 21:15:13.574898 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:13.574854 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5qxs8" podStartSLOduration=251.060262359 podStartE2EDuration="4m12.574840051s" totalImagesPullingTime="1.514577692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:15:13.573974368 +0000 UTC m=+287.317054368" watchObservedRunningTime="2026-06-11 21:15:13.574840051 +0000 UTC m=+287.317920108" Jun 11 21:15:14.751593 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:14.751561 2563 patch_prober.go:28] interesting pod/image-registry-84589dccd5-zrb5r container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 21:15:14.751945 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:14.751623 2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" podUID="15b77cd7-0a18-40d8-8c07-2dd97a915b5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 21:15:14.751945 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:14.751664 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:15:14.752158 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:14.752130 2563 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"88077ea501255de47db0c5ba37aa941ee2e001490d000818d1dc3078459359a0"} pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" containerMessage="Container registry failed liveness probe, will be restarted" Jun 11 21:15:14.755526 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:14.755496 2563 patch_prober.go:28] interesting pod/image-registry-84589dccd5-zrb5r container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 21:15:14.755628 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:14.755540 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" podUID="15b77cd7-0a18-40d8-8c07-2dd97a915b5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 21:15:17.713246 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:17.713210 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-566f7cc5bb-p4wjz_d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2/metrics-server/0.log" Jun 11 21:15:18.112799 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:18.112767 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-67vf2_e41a1472-8802-45ed-a95d-88814c60cdd1/init-textfile/0.log" Jun 11 21:15:18.314276 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:18.314246 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-67vf2_e41a1472-8802-45ed-a95d-88814c60cdd1/node-exporter/0.log" Jun 11 21:15:18.513393 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:18.513288 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-67vf2_e41a1472-8802-45ed-a95d-88814c60cdd1/kube-rbac-proxy/0.log" Jun 11 21:15:20.512917 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:20.512885 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a727a830-51f9-4fb8-bbd3-498269c1f3ac/init-config-reloader/0.log" Jun 11 21:15:20.715284 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:20.715253 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a727a830-51f9-4fb8-bbd3-498269c1f3ac/prometheus/0.log" Jun 11 21:15:20.913943 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:20.913913 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a727a830-51f9-4fb8-bbd3-498269c1f3ac/config-reloader/0.log" Jun 11 21:15:21.112981 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:21.112956 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a727a830-51f9-4fb8-bbd3-498269c1f3ac/thanos-sidecar/0.log" Jun 11 21:15:21.314696 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:21.314672 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a727a830-51f9-4fb8-bbd3-498269c1f3ac/kube-rbac-proxy-web/0.log" Jun 11 21:15:21.513014 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:21.512985 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a727a830-51f9-4fb8-bbd3-498269c1f3ac/kube-rbac-proxy/0.log" Jun 11 21:15:21.713906 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:21.713827 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a727a830-51f9-4fb8-bbd3-498269c1f3ac/kube-rbac-proxy-thanos/0.log" Jun 11 21:15:23.563584 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:23.563554 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5qxs8" Jun 11 21:15:23.913624 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:23.913544 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:15:24.114333 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:24.114301 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/3.log" Jun 11 21:15:24.756246 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:24.756207 2563 patch_prober.go:28] interesting pod/image-registry-84589dccd5-zrb5r container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 21:15:24.756709 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:24.756276 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" podUID="15b77cd7-0a18-40d8-8c07-2dd97a915b5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 21:15:25.313252 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:25.313222 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5qxs8_faf3cd10-7238-4409-bf88-932cd0b1c14e/dns/0.log" Jun 11 21:15:25.513412 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:25.513369 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5qxs8_faf3cd10-7238-4409-bf88-932cd0b1c14e/kube-rbac-proxy/0.log" Jun 11 21:15:26.113659 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:26.113631 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9wcfm_a76fe3c1-133a-4c0d-b36f-4f309a013579/dns-node-resolver/0.log" Jun 11 21:15:26.560746 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:26.560720 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:15:26.561373 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:26.561339 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:15:26.575158 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:26.575131 2563 kubelet.go:1648] "Image garbage collection succeeded" Jun 11 21:15:26.913436 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:26.913369 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-84589dccd5-zrb5r_15b77cd7-0a18-40d8-8c07-2dd97a915b5e/registry/0.log" Jun 11 21:15:27.513357 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:27.513331 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vb7pc_23d74d65-50c4-45b2-bfb0-33bd224279c8/node-ca/0.log" Jun 11 21:15:27.713811 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:27.713756 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d798db97b-wnl55_e4baf798-247a-4237-a103-aa1dae1b1557/router/0.log" Jun 11 21:15:28.318429 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:28.318396 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zvmqc_99fddcb2-b296-415e-94a9-15a40dc7462d/serve-healthcheck-canary/0.log" Jun 11 21:15:31.574755 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:31.574720 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:15:31.578529 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:31.578506 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-566f7cc5bb-p4wjz" Jun 11 21:15:34.756633 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:34.756605 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:15:39.770853 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:39.770801 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" podUID="15b77cd7-0a18-40d8-8c07-2dd97a915b5e" containerName="registry" containerID="cri-o://88077ea501255de47db0c5ba37aa941ee2e001490d000818d1dc3078459359a0" gracePeriod=30 Jun 11 21:15:41.658154 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:41.658113 2563 generic.go:356] "Generic (PLEG): container finished" podID="15b77cd7-0a18-40d8-8c07-2dd97a915b5e" containerID="88077ea501255de47db0c5ba37aa941ee2e001490d000818d1dc3078459359a0" exitCode=0 Jun 11 21:15:41.658605 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:41.658183 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" event={"ID":"15b77cd7-0a18-40d8-8c07-2dd97a915b5e","Type":"ContainerDied","Data":"88077ea501255de47db0c5ba37aa941ee2e001490d000818d1dc3078459359a0"} Jun 11 21:15:41.658605 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:41.658251 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" event={"ID":"15b77cd7-0a18-40d8-8c07-2dd97a915b5e","Type":"ContainerStarted","Data":"0b5e0c99fcd25e7490399af01ffce09dbb66b4b49d7f83e514815533e9819108"} Jun 11 21:15:41.658605 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:41.658278 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:15:53.479808 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:53.479748 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:15:53.495417 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:53.495390 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:15:53.714693 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:15:53.714668 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:02.666106 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:02.666072 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84589dccd5-zrb5r" Jun 11 21:16:11.507155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.507110 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 21:16:11.507660 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.507392 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="prometheus" containerID="cri-o://4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe" gracePeriod=600 Jun 11 21:16:11.507660 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.507418 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="kube-rbac-proxy" containerID="cri-o://ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c" gracePeriod=600 Jun 11 21:16:11.507660 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.507455 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d" gracePeriod=600 Jun 11 21:16:11.507660 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.507469 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="kube-rbac-proxy-web" containerID="cri-o://3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9" gracePeriod=600 Jun 11 21:16:11.507660 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.507482 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="thanos-sidecar" containerID="cri-o://d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b" gracePeriod=600 Jun 11 21:16:11.507660 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.507550 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="config-reloader" containerID="cri-o://ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561" gracePeriod=600 Jun 11 21:16:11.760454 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.760396 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:11.766659 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.766634 2563 generic.go:356] "Generic (PLEG): container finished" podID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerID="7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d" exitCode=0 Jun 11 21:16:11.766659 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.766655 2563 generic.go:356] "Generic (PLEG): container finished" podID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerID="ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c" exitCode=0 Jun 11 21:16:11.766836 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.766664 2563 generic.go:356] "Generic (PLEG): container finished" podID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerID="3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9" exitCode=0 Jun 11 21:16:11.766836 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.766674 2563 generic.go:356] "Generic (PLEG): container finished" podID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerID="d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b" exitCode=0 Jun 11 21:16:11.766836 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.766684 2563 generic.go:356] "Generic (PLEG): container finished" podID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerID="ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561" exitCode=0 Jun 11 21:16:11.766836 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.766691 2563 generic.go:356] "Generic (PLEG): container finished" podID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerID="4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe" exitCode=0 Jun 11 21:16:11.766836 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.766695 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a727a830-51f9-4fb8-bbd3-498269c1f3ac","Type":"ContainerDied","Data":"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d"} Jun 11 21:16:11.766836 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.766762 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a727a830-51f9-4fb8-bbd3-498269c1f3ac","Type":"ContainerDied","Data":"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c"} Jun 11 21:16:11.766836 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.766705 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:11.766836 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.766790 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a727a830-51f9-4fb8-bbd3-498269c1f3ac","Type":"ContainerDied","Data":"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9"} Jun 11 21:16:11.766836 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.766809 2563 scope.go:117] "RemoveContainer" containerID="7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d" Jun 11 21:16:11.766836 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.766821 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a727a830-51f9-4fb8-bbd3-498269c1f3ac","Type":"ContainerDied","Data":"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b"} Jun 11 21:16:11.767224 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.766848 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a727a830-51f9-4fb8-bbd3-498269c1f3ac","Type":"ContainerDied","Data":"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561"} Jun 11 21:16:11.767224 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.766907 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a727a830-51f9-4fb8-bbd3-498269c1f3ac","Type":"ContainerDied","Data":"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe"} Jun 11 21:16:11.767224 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.766964 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a727a830-51f9-4fb8-bbd3-498269c1f3ac","Type":"ContainerDied","Data":"5bf833021c54ec193080ff2539f440b16544cf38c85a1fee65a3be400ff64dee"} Jun 11 21:16:11.773871 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.773844 2563 scope.go:117] "RemoveContainer" containerID="ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c" Jun 11 21:16:11.780394 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.780375 2563 scope.go:117] "RemoveContainer" containerID="3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9" Jun 11 21:16:11.787204 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.787185 2563 scope.go:117] "RemoveContainer" containerID="d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b" Jun 11 21:16:11.795466 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.795336 2563 scope.go:117] "RemoveContainer" containerID="ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561" Jun 11 21:16:11.810594 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.810573 2563 scope.go:117] "RemoveContainer" containerID="4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe" Jun 11 21:16:11.818210 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.818194 2563 scope.go:117] "RemoveContainer" containerID="cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71" Jun 11 21:16:11.825106 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.825083 2563 scope.go:117] "RemoveContainer" containerID="7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d" Jun 11 21:16:11.825388 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:16:11.825367 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d\": container with ID starting with 7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d not found: ID does not exist" containerID="7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d" Jun 11 21:16:11.825459 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.825396 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d"} err="failed to get container status \"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d\": rpc error: code = NotFound desc = could not find container \"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d\": container with ID starting with 7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d not found: ID does not exist" Jun 11 21:16:11.825459 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.825420 2563 scope.go:117] "RemoveContainer" containerID="ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c" Jun 11 21:16:11.825666 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:16:11.825647 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c\": container with ID starting with ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c not found: ID does not exist" containerID="ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c" Jun 11 21:16:11.825722 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.825673 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c"} err="failed to get container status \"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c\": rpc error: code = NotFound desc = could not find container \"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c\": container with ID starting with ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c not found: ID does not exist" Jun 11 21:16:11.825722 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.825690 2563 scope.go:117] "RemoveContainer" containerID="3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9" Jun 11 21:16:11.825961 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:16:11.825927 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9\": container with ID starting with 3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9 not found: ID does not exist" containerID="3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9" Jun 11 21:16:11.826011 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.825963 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9"} err="failed to get container status \"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9\": rpc error: code = NotFound desc = could not find container \"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9\": container with ID starting with 3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9 not found: ID does not exist" Jun 11 21:16:11.826011 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.825986 2563 scope.go:117] "RemoveContainer" containerID="d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b" Jun 11 21:16:11.826320 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:16:11.826298 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b\": container with ID starting with d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b not found: ID does not exist" containerID="d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b" Jun 11 21:16:11.826411 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.826327 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b"} err="failed to get container status \"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b\": rpc error: code = NotFound desc = could not find container \"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b\": container with ID starting with d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b not found: ID does not exist" Jun 11 21:16:11.826411 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.826348 2563 scope.go:117] "RemoveContainer" containerID="ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561" Jun 11 21:16:11.826583 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:16:11.826565 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561\": container with ID starting with ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561 not found: ID does not exist" containerID="ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561" Jun 11 21:16:11.826662 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.826590 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561"} err="failed to get container status \"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561\": rpc error: code = NotFound desc = could not find container \"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561\": container with ID starting with ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561 not found: ID does not exist" Jun 11 21:16:11.826662 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.826608 2563 scope.go:117] "RemoveContainer" containerID="4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe" Jun 11 21:16:11.826864 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:16:11.826846 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe\": container with ID starting with 4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe not found: ID does not exist" containerID="4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe" Jun 11 21:16:11.826921 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.826868 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe"} err="failed to get container status \"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe\": rpc error: code = NotFound desc = could not find container \"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe\": container with ID starting with 4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe not found: ID does not exist" Jun 11 21:16:11.826921 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.826882 2563 scope.go:117] "RemoveContainer" containerID="cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71" Jun 11 21:16:11.827129 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:16:11.827112 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71\": container with ID starting with cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71 not found: ID does not exist" containerID="cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71" Jun 11 21:16:11.827206 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.827135 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71"} err="failed to get container status \"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71\": rpc error: code = NotFound desc = could not find container \"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71\": container with ID starting with cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71 not found: ID does not exist" Jun 11 21:16:11.827206 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.827162 2563 scope.go:117] "RemoveContainer" containerID="7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d" Jun 11 21:16:11.827410 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.827392 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d"} err="failed to get container status \"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d\": rpc error: code = NotFound desc = could not find container \"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d\": container with ID starting with 7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d not found: ID does not exist" Jun 11 21:16:11.827410 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.827410 2563 scope.go:117] "RemoveContainer" containerID="ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c" Jun 11 21:16:11.827673 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.827655 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c"} err="failed to get container status \"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c\": rpc error: code = NotFound desc = could not find container \"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c\": container with ID starting with ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c not found: ID does not exist" Jun 11 21:16:11.827717 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.827674 2563 scope.go:117] "RemoveContainer" containerID="3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9" Jun 11 21:16:11.827921 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.827903 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9"} err="failed to get container status \"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9\": rpc error: code = NotFound desc = could not find container \"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9\": container with ID starting with 3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9 not found: ID does not exist" Jun 11 21:16:11.827992 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.827922 2563 scope.go:117] "RemoveContainer" containerID="d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b" Jun 11 21:16:11.828193 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.828175 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b"} err="failed to get container status \"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b\": rpc error: code = NotFound desc = could not find container \"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b\": container with ID starting with d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b not found: ID does not exist" Jun 11 21:16:11.828245 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.828194 2563 scope.go:117] "RemoveContainer" containerID="ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561" Jun 11 21:16:11.828402 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.828386 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561"} err="failed to get container status \"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561\": rpc error: code = NotFound desc = could not find container \"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561\": container with ID starting with ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561 not found: ID does not exist" Jun 11 21:16:11.828456 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.828402 2563 scope.go:117] "RemoveContainer" containerID="4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe" Jun 11 21:16:11.828616 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.828598 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe"} err="failed to get container status \"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe\": rpc error: code = NotFound desc = could not find container \"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe\": container with ID starting with 4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe not found: ID does not exist" Jun 11 21:16:11.828687 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.828618 2563 scope.go:117] "RemoveContainer" containerID="cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71" Jun 11 21:16:11.828874 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.828856 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71"} err="failed to get container status \"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71\": rpc error: code = NotFound desc = could not find container \"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71\": container with ID starting with cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71 not found: ID does not exist" Jun 11 21:16:11.828918 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.828874 2563 scope.go:117] "RemoveContainer" containerID="7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d" Jun 11 21:16:11.829178 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.829159 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d"} err="failed to get container status \"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d\": rpc error: code = NotFound desc = could not find container \"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d\": container with ID starting with 7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d not found: ID does not exist" Jun 11 21:16:11.829255 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.829180 2563 scope.go:117] "RemoveContainer" containerID="ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c" Jun 11 21:16:11.829412 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.829394 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c"} err="failed to get container status \"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c\": rpc error: code = NotFound desc = could not find container \"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c\": container with ID starting with ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c not found: ID does not exist" Jun 11 21:16:11.829478 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.829413 2563 scope.go:117] "RemoveContainer" containerID="3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9" Jun 11 21:16:11.829629 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.829613 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9"} err="failed to get container status \"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9\": rpc error: code = NotFound desc = could not find container \"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9\": container with ID starting with 3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9 not found: ID does not exist" Jun 11 21:16:11.829706 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.829630 2563 scope.go:117] "RemoveContainer" containerID="d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b" Jun 11 21:16:11.829861 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.829844 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b"} err="failed to get container status \"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b\": rpc error: code = NotFound desc = could not find container \"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b\": container with ID starting with d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b not found: ID does not exist" Jun 11 21:16:11.829903 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.829862 2563 scope.go:117] "RemoveContainer" containerID="ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561" Jun 11 21:16:11.830073 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.830055 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561"} err="failed to get container status \"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561\": rpc error: code = NotFound desc = could not find container \"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561\": container with ID starting with ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561 not found: ID does not exist" Jun 11 21:16:11.830147 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.830074 2563 scope.go:117] "RemoveContainer" containerID="4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe" Jun 11 21:16:11.830316 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.830300 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe"} err="failed to get container status \"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe\": rpc error: code = NotFound desc = could not find container \"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe\": container with ID starting with 4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe not found: ID does not exist" Jun 11 21:16:11.830388 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.830317 2563 scope.go:117] "RemoveContainer" containerID="cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71" Jun 11 21:16:11.830526 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.830509 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71"} err="failed to get container status \"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71\": rpc error: code = NotFound desc = could not find container \"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71\": container with ID starting with cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71 not found: ID does not exist" Jun 11 21:16:11.830593 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.830527 2563 scope.go:117] "RemoveContainer" containerID="7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d" Jun 11 21:16:11.830731 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.830707 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d"} err="failed to get container status \"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d\": rpc error: code = NotFound desc = could not find container \"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d\": container with ID starting with 7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d not found: ID does not exist" Jun 11 21:16:11.830731 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.830725 2563 scope.go:117] "RemoveContainer" containerID="ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c" Jun 11 21:16:11.830911 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.830895 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c"} err="failed to get container status \"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c\": rpc error: code = NotFound desc = could not find container \"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c\": container with ID starting with ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c not found: ID does not exist" Jun 11 21:16:11.830971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.830912 2563 scope.go:117] "RemoveContainer" containerID="3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9" Jun 11 21:16:11.831182 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.831162 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9"} err="failed to get container status \"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9\": rpc error: code = NotFound desc = could not find container \"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9\": container with ID starting with 3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9 not found: ID does not exist" Jun 11 21:16:11.831260 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.831182 2563 scope.go:117] "RemoveContainer" containerID="d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b" Jun 11 21:16:11.831415 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.831398 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b"} err="failed to get container status \"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b\": rpc error: code = NotFound desc = could not find container \"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b\": container with ID starting with d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b not found: ID does not exist" Jun 11 21:16:11.831481 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.831417 2563 scope.go:117] "RemoveContainer" containerID="ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561" Jun 11 21:16:11.831630 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.831608 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561"} err="failed to get container status \"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561\": rpc error: code = NotFound desc = could not find container \"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561\": container with ID starting with ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561 not found: ID does not exist" Jun 11 21:16:11.831689 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.831629 2563 scope.go:117] "RemoveContainer" containerID="4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe" Jun 11 21:16:11.831837 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.831821 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe"} err="failed to get container status \"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe\": rpc error: code = NotFound desc = could not find container \"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe\": container with ID starting with 4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe not found: ID does not exist" Jun 11 21:16:11.831905 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.831838 2563 scope.go:117] "RemoveContainer" containerID="cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71" Jun 11 21:16:11.832067 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.832049 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71"} err="failed to get container status \"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71\": rpc error: code = NotFound desc = could not find container \"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71\": container with ID starting with cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71 not found: ID does not exist" Jun 11 21:16:11.832110 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.832068 2563 scope.go:117] "RemoveContainer" containerID="7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d" Jun 11 21:16:11.832303 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.832285 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d"} err="failed to get container status \"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d\": rpc error: code = NotFound desc = could not find container \"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d\": container with ID starting with 7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d not found: ID does not exist" Jun 11 21:16:11.832348 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.832304 2563 scope.go:117] "RemoveContainer" containerID="ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c" Jun 11 21:16:11.832512 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.832497 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c"} err="failed to get container status \"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c\": rpc error: code = NotFound desc = could not find container \"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c\": container with ID starting with ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c not found: ID does not exist" Jun 11 21:16:11.832512 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.832511 2563 scope.go:117] "RemoveContainer" containerID="3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9" Jun 11 21:16:11.832719 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.832700 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9"} err="failed to get container status \"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9\": rpc error: code = NotFound desc = could not find container \"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9\": container with ID starting with 3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9 not found: ID does not exist" Jun 11 21:16:11.832785 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.832721 2563 scope.go:117] "RemoveContainer" containerID="d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b" Jun 11 21:16:11.832954 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.832936 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b"} err="failed to get container status \"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b\": rpc error: code = NotFound desc = could not find container \"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b\": container with ID starting with d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b not found: ID does not exist" Jun 11 21:16:11.833011 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.832954 2563 scope.go:117] "RemoveContainer" containerID="ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561" Jun 11 21:16:11.833171 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.833152 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561"} err="failed to get container status \"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561\": rpc error: code = NotFound desc = could not find container \"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561\": container with ID starting with ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561 not found: ID does not exist" Jun 11 21:16:11.833228 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.833171 2563 scope.go:117] "RemoveContainer" containerID="4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe" Jun 11 21:16:11.833414 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.833395 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe"} err="failed to get container status \"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe\": rpc error: code = NotFound desc = could not find container \"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe\": container with ID starting with 4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe not found: ID does not exist" Jun 11 21:16:11.833476 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.833415 2563 scope.go:117] "RemoveContainer" containerID="cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71" Jun 11 21:16:11.833671 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.833654 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71"} err="failed to get container status \"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71\": rpc error: code = NotFound desc = could not find container \"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71\": container with ID starting with cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71 not found: ID does not exist" Jun 11 21:16:11.833719 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.833673 2563 scope.go:117] "RemoveContainer" containerID="7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d" Jun 11 21:16:11.833934 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.833916 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d"} err="failed to get container status \"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d\": rpc error: code = NotFound desc = could not find container \"7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d\": container with ID starting with 7dddf96d34d8db0089e55b93b2653b74c6ac52e2a8283b0d82848688fe91d63d not found: ID does not exist" Jun 11 21:16:11.833934 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.833933 2563 scope.go:117] "RemoveContainer" containerID="ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c" Jun 11 21:16:11.834171 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.834154 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c"} err="failed to get container status \"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c\": rpc error: code = NotFound desc = could not find container \"ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c\": container with ID starting with ead135d974e2720fe8ea3d038bd3f37b2b760059b47d4df83a26036398cd5c5c not found: ID does not exist" Jun 11 21:16:11.834216 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.834171 2563 scope.go:117] "RemoveContainer" containerID="3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9" Jun 11 21:16:11.834391 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.834373 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9"} err="failed to get container status \"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9\": rpc error: code = NotFound desc = could not find container \"3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9\": container with ID starting with 3107cf298999afd6cbab96670c6c4bd1352c2c31de897d0824faf16f3b53a1d9 not found: ID does not exist" Jun 11 21:16:11.834391 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.834390 2563 scope.go:117] "RemoveContainer" containerID="d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b" Jun 11 21:16:11.834596 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.834579 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b"} err="failed to get container status \"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b\": rpc error: code = NotFound desc = could not find container \"d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b\": container with ID starting with d65cd3d07b62ecb9b855e8225346e2a1fd67798a1ab067c38960ca5d4aca683b not found: ID does not exist" Jun 11 21:16:11.834596 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.834596 2563 scope.go:117] "RemoveContainer" containerID="ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561" Jun 11 21:16:11.834821 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.834805 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561"} err="failed to get container status \"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561\": rpc error: code = NotFound desc = could not find container \"ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561\": container with ID starting with ec777c018578b8978ed933c4f37acc97f8f9dd4239aa70ae607d31db9eab7561 not found: ID does not exist" Jun 11 21:16:11.834876 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.834821 2563 scope.go:117] "RemoveContainer" containerID="4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe" Jun 11 21:16:11.835047 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.835014 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe"} err="failed to get container status \"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe\": rpc error: code = NotFound desc = could not find container \"4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe\": container with ID starting with 4ec834fc40993faa43b1280d2e6e63b99eb42d759d2dcf7761bd1137142d9abe not found: ID does not exist" Jun 11 21:16:11.835093 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.835047 2563 scope.go:117] "RemoveContainer" containerID="cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71" Jun 11 21:16:11.835297 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.835273 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71"} err="failed to get container status \"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71\": rpc error: code = NotFound desc = could not find container \"cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71\": container with ID starting with cbb96b8dbe7beb9450cdc7d5ec15c664de17015b4e70fff5b1d0e3f8d36a1b71 not found: ID does not exist" Jun 11 21:16:11.915626 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.915595 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-2\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.915759 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.915643 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-tls\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.915759 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.915662 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-metrics-client-ca\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.915759 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.915678 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-config\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.915759 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.915705 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.915759 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.915722 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-thanos-prometheus-http-client-file\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.915759 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.915743 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.916088 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.915773 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-grpc-tls\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.916088 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.915800 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj5cm\" (UniqueName: \"kubernetes.io/projected/a727a830-51f9-4fb8-bbd3-498269c1f3ac-kube-api-access-nj5cm\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.916088 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.915829 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-trusted-ca-bundle\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.916088 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.915856 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a727a830-51f9-4fb8-bbd3-498269c1f3ac-tls-assets\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.916088 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.915884 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-0\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.916088 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.915921 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-1\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.916088 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.915952 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-serving-certs-ca-bundle\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.916088 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.915978 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a727a830-51f9-4fb8-bbd3-498269c1f3ac-config-out\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.916088 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.916006 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-metrics-client-certs\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.916088 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.916054 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:16:11.916088 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.916073 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:16:11.916088 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.916079 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-kube-rbac-proxy\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.916676 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.916129 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-kubelet-serving-ca-bundle\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.916676 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.916168 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-db\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.916676 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.916254 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-web-config\") pod \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\" (UID: \"a727a830-51f9-4fb8-bbd3-498269c1f3ac\") " Jun 11 21:16:11.916676 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.916632 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:11.916676 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.916653 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-metrics-client-ca\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:11.916932 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.916878 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:16:11.917312 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.917284 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:16:11.918152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.918121 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:16:11.918717 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.918594 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:16:11.918717 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.918682 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-config" (OuterVolumeSpecName: "config") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:16:11.919444 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.919240 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:16:11.919444 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.919273 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 21:16:11.919589 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.919380 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:16:11.919589 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.919573 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a727a830-51f9-4fb8-bbd3-498269c1f3ac-kube-api-access-nj5cm" (OuterVolumeSpecName: "kube-api-access-nj5cm") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "kube-api-access-nj5cm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:16:11.919699 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.919678 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:16:11.920068 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.920041 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:16:11.920186 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.920062 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:16:11.920186 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.920046 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a727a830-51f9-4fb8-bbd3-498269c1f3ac-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:16:11.920409 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.920378 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:16:11.921042 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.921000 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:16:11.921152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.921131 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:16:11.921495 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.921469 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a727a830-51f9-4fb8-bbd3-498269c1f3ac-config-out" (OuterVolumeSpecName: "config-out") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 21:16:11.929705 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:11.929676 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-web-config" (OuterVolumeSpecName: "web-config") pod "a727a830-51f9-4fb8-bbd3-498269c1f3ac" (UID: "a727a830-51f9-4fb8-bbd3-498269c1f3ac"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:16:12.017467 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017384 2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-web-config\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017467 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017413 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017467 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017424 2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-config\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017467 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017435 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017467 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017446 2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017467 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017456 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017467 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017464 2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-grpc-tls\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017467 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017473 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj5cm\" (UniqueName: \"kubernetes.io/projected/a727a830-51f9-4fb8-bbd3-498269c1f3ac-kube-api-access-nj5cm\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017481 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017489 2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a727a830-51f9-4fb8-bbd3-498269c1f3ac-tls-assets\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017498 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017506 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017515 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017523 2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a727a830-51f9-4fb8-bbd3-498269c1f3ac-config-out\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017531 2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-metrics-client-certs\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017540 2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a727a830-51f9-4fb8-bbd3-498269c1f3ac-secret-kube-rbac-proxy\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017548 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a727a830-51f9-4fb8-bbd3-498269c1f3ac-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.017803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.017558 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a727a830-51f9-4fb8-bbd3-498269c1f3ac-prometheus-k8s-db\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:16:12.091057 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.091030 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 21:16:12.099745 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.099719 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 21:16:12.127095 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127061 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 21:16:12.127337 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127324 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="config-reloader" Jun 11 21:16:12.127380 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127339 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="config-reloader" Jun 11 21:16:12.127380 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127350 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="init-config-reloader" Jun 11 21:16:12.127380 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127356 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="init-config-reloader" Jun 11 21:16:12.127380 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127361 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="kube-rbac-proxy-thanos" Jun 11 21:16:12.127380 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127367 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="kube-rbac-proxy-thanos" Jun 11 21:16:12.127380 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127374 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="kube-rbac-proxy" Jun 11 21:16:12.127380 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127379 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="kube-rbac-proxy" Jun 11 21:16:12.127577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127390 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f6b62f4-15c4-468b-9858-cfbcf2518e55" containerName="registry" Jun 11 21:16:12.127577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127395 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f6b62f4-15c4-468b-9858-cfbcf2518e55" containerName="registry" Jun 11 21:16:12.127577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127402 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="prometheus" Jun 11 21:16:12.127577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127408 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="prometheus" Jun 11 21:16:12.127577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127417 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="thanos-sidecar" Jun 11 21:16:12.127577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127421 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="thanos-sidecar" Jun 11 21:16:12.127577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127428 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="kube-rbac-proxy-web" Jun 11 21:16:12.127577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127434 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="kube-rbac-proxy-web" Jun 11 21:16:12.127577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127472 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="kube-rbac-proxy-web" Jun 11 21:16:12.127577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127480 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="kube-rbac-proxy" Jun 11 21:16:12.127577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127485 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="kube-rbac-proxy-thanos" Jun 11 21:16:12.127577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127490 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f6b62f4-15c4-468b-9858-cfbcf2518e55" containerName="registry" Jun 11 21:16:12.127577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127497 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="thanos-sidecar" Jun 11 21:16:12.127577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127503 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="prometheus" Jun 11 21:16:12.127577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.127508 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" containerName="config-reloader" Jun 11 21:16:12.132505 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.132489 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.134806 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.134785 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 11 21:16:12.134922 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.134786 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 11 21:16:12.134922 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.134871 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-nglp3a1a03ua\"" Jun 11 21:16:12.135232 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.135215 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 11 21:16:12.135315 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.135236 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 11 21:16:12.135315 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.135255 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-svl29\"" Jun 11 21:16:12.135315 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.135276 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 11 21:16:12.135315 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.135312 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 11 21:16:12.135507 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.135359 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 11 21:16:12.135507 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.135412 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 11 21:16:12.135640 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.135626 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 11 21:16:12.135702 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.135627 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 11 21:16:12.135752 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.135709 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 11 21:16:12.136118 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.136092 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 11 21:16:12.138455 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.138437 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 11 21:16:12.140559 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.140539 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 11 21:16:12.148558 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.148537 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 21:16:12.220137 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220096 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220137 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220138 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220378 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220203 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220378 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220232 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl4w9\" (UniqueName: \"kubernetes.io/projected/20261d7f-29da-4d6f-9d71-2b0b30b31c43-kube-api-access-dl4w9\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220378 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220256 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20261d7f-29da-4d6f-9d71-2b0b30b31c43-config-out\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220378 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220298 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220378 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220315 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220378 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220334 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220378 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220355 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-config\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220378 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220376 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220780 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220397 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220780 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220414 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220780 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220431 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20261d7f-29da-4d6f-9d71-2b0b30b31c43-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220780 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220447 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/20261d7f-29da-4d6f-9d71-2b0b30b31c43-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220780 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220493 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220780 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220513 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220780 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220563 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-web-config\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220780 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220586 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220780 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220606 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.220780 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.220630 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.321637 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.321600 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.321824 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.321645 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.321824 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.321663 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-web-config\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.321824 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.321680 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.321824 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.321704 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322013 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.321829 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322013 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.321877 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322013 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.321909 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322013 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.321948 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322013 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.321973 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl4w9\" (UniqueName: \"kubernetes.io/projected/20261d7f-29da-4d6f-9d71-2b0b30b31c43-kube-api-access-dl4w9\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322013 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.321998 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20261d7f-29da-4d6f-9d71-2b0b30b31c43-config-out\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322346 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.322080 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322346 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.322108 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322346 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.322142 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322346 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.322178 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-config\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322346 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.322205 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322346 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.322231 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322346 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.322259 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322346 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.322286 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20261d7f-29da-4d6f-9d71-2b0b30b31c43-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322346 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.322310 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/20261d7f-29da-4d6f-9d71-2b0b30b31c43-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322766 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.322546 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322766 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.322669 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/20261d7f-29da-4d6f-9d71-2b0b30b31c43-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.322952 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.322924 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.323227 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.323204 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.323304 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.323233 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.323817 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.323502 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.323932 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.323844 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.324790 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.324760 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.324887 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.324821 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-web-config\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.324887 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.324858 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.325723 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.325697 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.325952 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.325926 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/20261d7f-29da-4d6f-9d71-2b0b30b31c43-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.326099 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.326081 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.326279 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.326261 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-config\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.326380 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.326366 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20261d7f-29da-4d6f-9d71-2b0b30b31c43-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.326987 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.326970 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20261d7f-29da-4d6f-9d71-2b0b30b31c43-config-out\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.329282 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.329254 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.329412 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.329396 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.329459 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.329422 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/20261d7f-29da-4d6f-9d71-2b0b30b31c43-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.332237 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.332219 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl4w9\" (UniqueName: \"kubernetes.io/projected/20261d7f-29da-4d6f-9d71-2b0b30b31c43-kube-api-access-dl4w9\") pod \"prometheus-k8s-0\" (UID: \"20261d7f-29da-4d6f-9d71-2b0b30b31c43\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.442190 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.442152 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:16:12.577855 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.577695 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 21:16:12.582414 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:16:12.582380 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20261d7f_29da_4d6f_9d71_2b0b30b31c43.slice/crio-a800d4953869380ffb7ccddfeee19634b98d6ac7b02c7b4f377fd1c241e076d3 WatchSource:0}: Error finding container a800d4953869380ffb7ccddfeee19634b98d6ac7b02c7b4f377fd1c241e076d3: Status 404 returned error can't find the container with id a800d4953869380ffb7ccddfeee19634b98d6ac7b02c7b4f377fd1c241e076d3 Jun 11 21:16:12.615480 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.615455 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a727a830-51f9-4fb8-bbd3-498269c1f3ac" path="/var/lib/kubelet/pods/a727a830-51f9-4fb8-bbd3-498269c1f3ac/volumes" Jun 11 21:16:12.772975 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.772939 2563 generic.go:356] "Generic (PLEG): container finished" podID="20261d7f-29da-4d6f-9d71-2b0b30b31c43" containerID="ebc7e1786d138ad6724b22266170702e2607b0581c06dfd1c28b086dfdc54031" exitCode=0 Jun 11 21:16:12.773182 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.773052 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20261d7f-29da-4d6f-9d71-2b0b30b31c43","Type":"ContainerDied","Data":"ebc7e1786d138ad6724b22266170702e2607b0581c06dfd1c28b086dfdc54031"} Jun 11 21:16:12.773182 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:12.773114 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20261d7f-29da-4d6f-9d71-2b0b30b31c43","Type":"ContainerStarted","Data":"a800d4953869380ffb7ccddfeee19634b98d6ac7b02c7b4f377fd1c241e076d3"} Jun 11 21:16:13.779873 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:13.779780 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20261d7f-29da-4d6f-9d71-2b0b30b31c43","Type":"ContainerStarted","Data":"c946c8e41b133c5ee04806bd4f90e77d1e9bd66a6a686a2b2159b30fb19e8a91"} Jun 11 21:16:13.780349 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:13.779887 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20261d7f-29da-4d6f-9d71-2b0b30b31c43","Type":"ContainerStarted","Data":"2543fa5647c93ac3830924bc858148e5885098405d7945a776fcd5277f1f02c0"} Jun 11 21:16:13.780349 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:13.779927 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20261d7f-29da-4d6f-9d71-2b0b30b31c43","Type":"ContainerStarted","Data":"a9d65e356195eca0fea4f082fc74167b4027080501265c3121f9a70d527afa6f"} Jun 11 21:16:13.780349 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:13.779965 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20261d7f-29da-4d6f-9d71-2b0b30b31c43","Type":"ContainerStarted","Data":"12bcae6cc476a8c3c16370c85d305c26ec30352acaf6bcd49c12a02efe456e54"} Jun 11 21:16:13.780349 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:13.780007 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20261d7f-29da-4d6f-9d71-2b0b30b31c43","Type":"ContainerStarted","Data":"46a5de3fdacaf6fcaa7713c9bf7560e6da58643b0c4368b8a18e1279a83b243b"} Jun 11 21:16:13.780349 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:13.780066 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"20261d7f-29da-4d6f-9d71-2b0b30b31c43","Type":"ContainerStarted","Data":"902ac0cec1a28478023892cf023359253b949657d8881bb58f3c2b44bdab1cad"} Jun 11 21:16:13.807708 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:13.807658 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.7843129960000002 podStartE2EDuration="1.80764146s" totalImagesPullingTime="0s" totalInitContainerRuntime="23.328464ms" isStatefulPod=false podCreationTimestamp="2026-06-11 21:16:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:16:13.804827074 +0000 UTC m=+347.547907062" watchObservedRunningTime="2026-06-11 21:16:13.80764146 +0000 UTC m=+347.550721448" Jun 11 21:16:17.443314 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:16:17.443229 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:17:12.443331 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:17:12.443296 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:17:12.458550 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:17:12.458529 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:17:13.000265 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:17:13.000233 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 21:19:19.378592 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.378508 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn"] Jun 11 21:19:19.384567 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.384537 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.387776 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.387747 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 11 21:19:19.387904 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.387794 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 11 21:19:19.388196 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.388183 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 11 21:19:19.388405 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.388387 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9ssfh\"" Jun 11 21:19:19.388485 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.388425 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 11 21:19:19.397599 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.397576 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn"] Jun 11 21:19:19.491793 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.491756 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4b01aa7-5f6b-4f71-870d-d9043ce38c61-webhook-cert\") pod \"opendatahub-operator-controller-manager-66c5c8d746-jpksn\" (UID: \"a4b01aa7-5f6b-4f71-870d-d9043ce38c61\") " pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.491979 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.491811 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sqr6\" (UniqueName: \"kubernetes.io/projected/a4b01aa7-5f6b-4f71-870d-d9043ce38c61-kube-api-access-5sqr6\") pod \"opendatahub-operator-controller-manager-66c5c8d746-jpksn\" (UID: \"a4b01aa7-5f6b-4f71-870d-d9043ce38c61\") " pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.491979 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.491842 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a4b01aa7-5f6b-4f71-870d-d9043ce38c61-manifests\") pod \"opendatahub-operator-controller-manager-66c5c8d746-jpksn\" (UID: \"a4b01aa7-5f6b-4f71-870d-d9043ce38c61\") " pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.491979 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.491868 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4b01aa7-5f6b-4f71-870d-d9043ce38c61-tmp\") pod \"opendatahub-operator-controller-manager-66c5c8d746-jpksn\" (UID: \"a4b01aa7-5f6b-4f71-870d-d9043ce38c61\") " pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.491979 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.491888 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a4b01aa7-5f6b-4f71-870d-d9043ce38c61-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66c5c8d746-jpksn\" (UID: \"a4b01aa7-5f6b-4f71-870d-d9043ce38c61\") " pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.592810 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.592781 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4b01aa7-5f6b-4f71-870d-d9043ce38c61-webhook-cert\") pod \"opendatahub-operator-controller-manager-66c5c8d746-jpksn\" (UID: \"a4b01aa7-5f6b-4f71-870d-d9043ce38c61\") " pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.592987 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.592826 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sqr6\" (UniqueName: \"kubernetes.io/projected/a4b01aa7-5f6b-4f71-870d-d9043ce38c61-kube-api-access-5sqr6\") pod \"opendatahub-operator-controller-manager-66c5c8d746-jpksn\" (UID: \"a4b01aa7-5f6b-4f71-870d-d9043ce38c61\") " pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.592987 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.592855 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a4b01aa7-5f6b-4f71-870d-d9043ce38c61-manifests\") pod \"opendatahub-operator-controller-manager-66c5c8d746-jpksn\" (UID: \"a4b01aa7-5f6b-4f71-870d-d9043ce38c61\") " pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.592987 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.592881 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4b01aa7-5f6b-4f71-870d-d9043ce38c61-tmp\") pod \"opendatahub-operator-controller-manager-66c5c8d746-jpksn\" (UID: \"a4b01aa7-5f6b-4f71-870d-d9043ce38c61\") " pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.592987 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.592904 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a4b01aa7-5f6b-4f71-870d-d9043ce38c61-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66c5c8d746-jpksn\" (UID: \"a4b01aa7-5f6b-4f71-870d-d9043ce38c61\") " pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.593370 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.593339 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a4b01aa7-5f6b-4f71-870d-d9043ce38c61-manifests\") pod \"opendatahub-operator-controller-manager-66c5c8d746-jpksn\" (UID: \"a4b01aa7-5f6b-4f71-870d-d9043ce38c61\") " pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.595152 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.595116 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4b01aa7-5f6b-4f71-870d-d9043ce38c61-tmp\") pod \"opendatahub-operator-controller-manager-66c5c8d746-jpksn\" (UID: \"a4b01aa7-5f6b-4f71-870d-d9043ce38c61\") " pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.595287 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.595270 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4b01aa7-5f6b-4f71-870d-d9043ce38c61-webhook-cert\") pod \"opendatahub-operator-controller-manager-66c5c8d746-jpksn\" (UID: \"a4b01aa7-5f6b-4f71-870d-d9043ce38c61\") " pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.595370 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.595355 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a4b01aa7-5f6b-4f71-870d-d9043ce38c61-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66c5c8d746-jpksn\" (UID: \"a4b01aa7-5f6b-4f71-870d-d9043ce38c61\") " pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.600062 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.600045 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sqr6\" (UniqueName: \"kubernetes.io/projected/a4b01aa7-5f6b-4f71-870d-d9043ce38c61-kube-api-access-5sqr6\") pod \"opendatahub-operator-controller-manager-66c5c8d746-jpksn\" (UID: \"a4b01aa7-5f6b-4f71-870d-d9043ce38c61\") " pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.695288 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.695202 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:19.853959 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.853931 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn"] Jun 11 21:19:19.856915 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:19:19.856889 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4b01aa7_5f6b_4f71_870d_d9043ce38c61.slice/crio-c6649d28fe3b940e94973563ea6a78d7fb35613f32f7745754d5a235269d1814 WatchSource:0}: Error finding container c6649d28fe3b940e94973563ea6a78d7fb35613f32f7745754d5a235269d1814: Status 404 returned error can't find the container with id c6649d28fe3b940e94973563ea6a78d7fb35613f32f7745754d5a235269d1814 Jun 11 21:19:19.858626 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:19.858607 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 21:19:20.427717 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:20.427682 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" event={"ID":"a4b01aa7-5f6b-4f71-870d-d9043ce38c61","Type":"ContainerStarted","Data":"c6649d28fe3b940e94973563ea6a78d7fb35613f32f7745754d5a235269d1814"} Jun 11 21:19:23.443179 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:23.443146 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" event={"ID":"a4b01aa7-5f6b-4f71-870d-d9043ce38c61","Type":"ContainerStarted","Data":"11794042f669b6b493d0766bc2d9f0f8c98c4e5f3cc73f82482bc5734c98d0b0"} Jun 11 21:19:24.448796 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:24.448765 2563 generic.go:356] "Generic (PLEG): container finished" podID="a4b01aa7-5f6b-4f71-870d-d9043ce38c61" containerID="11794042f669b6b493d0766bc2d9f0f8c98c4e5f3cc73f82482bc5734c98d0b0" exitCode=0 Jun 11 21:19:24.449233 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:24.448815 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" event={"ID":"a4b01aa7-5f6b-4f71-870d-d9043ce38c61","Type":"ContainerDied","Data":"11794042f669b6b493d0766bc2d9f0f8c98c4e5f3cc73f82482bc5734c98d0b0"} Jun 11 21:19:25.454061 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:25.453999 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" event={"ID":"a4b01aa7-5f6b-4f71-870d-d9043ce38c61","Type":"ContainerStarted","Data":"27e58a3e3ccbb66dc9264e6f20f9f2750c11650e94250d707ded8df091f36b6d"} Jun 11 21:19:25.454449 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:25.454146 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:25.475409 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:25.475362 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" podStartSLOduration=2.097988301 podStartE2EDuration="6.475348611s" totalImagesPullingTime="3.370289209s" totalInitContainerRuntime="1.007071101s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:19:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:19:25.475050349 +0000 UTC m=+539.218130363" watchObservedRunningTime="2026-06-11 21:19:25.475348611 +0000 UTC m=+539.218428598" Jun 11 21:19:36.460708 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:36.460670 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-66c5c8d746-jpksn" Jun 11 21:19:44.140843 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:44.140805 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk"] Jun 11 21:19:44.155706 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:44.155677 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk"] Jun 11 21:19:44.155850 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:44.155815 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" Jun 11 21:19:44.158153 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:44.158129 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 11 21:19:44.158304 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:44.158166 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 11 21:19:44.158304 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:44.158214 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-64wjp\"" Jun 11 21:19:44.193221 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:44.193189 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pct72\" (UniqueName: \"kubernetes.io/projected/abd6e8a3-e983-47bb-a937-890763c5eb26-kube-api-access-pct72\") pod \"kube-auth-proxy-6c9dfcd5c9-64xrk\" (UID: \"abd6e8a3-e983-47bb-a937-890763c5eb26\") " pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" Jun 11 21:19:44.193371 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:44.193235 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abd6e8a3-e983-47bb-a937-890763c5eb26-tls-certs\") pod \"kube-auth-proxy-6c9dfcd5c9-64xrk\" (UID: \"abd6e8a3-e983-47bb-a937-890763c5eb26\") " pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" Jun 11 21:19:44.193371 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:44.193314 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abd6e8a3-e983-47bb-a937-890763c5eb26-tmp\") pod \"kube-auth-proxy-6c9dfcd5c9-64xrk\" (UID: \"abd6e8a3-e983-47bb-a937-890763c5eb26\") " pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" Jun 11 21:19:44.294081 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:44.294042 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abd6e8a3-e983-47bb-a937-890763c5eb26-tmp\") pod \"kube-auth-proxy-6c9dfcd5c9-64xrk\" (UID: \"abd6e8a3-e983-47bb-a937-890763c5eb26\") " pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" Jun 11 21:19:44.294238 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:44.294117 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pct72\" (UniqueName: \"kubernetes.io/projected/abd6e8a3-e983-47bb-a937-890763c5eb26-kube-api-access-pct72\") pod \"kube-auth-proxy-6c9dfcd5c9-64xrk\" (UID: \"abd6e8a3-e983-47bb-a937-890763c5eb26\") " pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" Jun 11 21:19:44.294238 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:44.294142 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abd6e8a3-e983-47bb-a937-890763c5eb26-tls-certs\") pod \"kube-auth-proxy-6c9dfcd5c9-64xrk\" (UID: \"abd6e8a3-e983-47bb-a937-890763c5eb26\") " pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" Jun 11 21:19:44.294349 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:19:44.294238 2563 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jun 11 21:19:44.294349 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:19:44.294302 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abd6e8a3-e983-47bb-a937-890763c5eb26-tls-certs podName:abd6e8a3-e983-47bb-a937-890763c5eb26 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:44.7942856 +0000 UTC m=+558.537365567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/abd6e8a3-e983-47bb-a937-890763c5eb26-tls-certs") pod "kube-auth-proxy-6c9dfcd5c9-64xrk" (UID: "abd6e8a3-e983-47bb-a937-890763c5eb26") : secret "kube-auth-proxy-tls" not found Jun 11 21:19:44.296227 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:44.296205 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abd6e8a3-e983-47bb-a937-890763c5eb26-tmp\") pod \"kube-auth-proxy-6c9dfcd5c9-64xrk\" (UID: \"abd6e8a3-e983-47bb-a937-890763c5eb26\") " pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" Jun 11 21:19:44.304444 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:44.304423 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pct72\" (UniqueName: \"kubernetes.io/projected/abd6e8a3-e983-47bb-a937-890763c5eb26-kube-api-access-pct72\") pod \"kube-auth-proxy-6c9dfcd5c9-64xrk\" (UID: \"abd6e8a3-e983-47bb-a937-890763c5eb26\") " pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" Jun 11 21:19:44.798841 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:44.798804 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abd6e8a3-e983-47bb-a937-890763c5eb26-tls-certs\") pod \"kube-auth-proxy-6c9dfcd5c9-64xrk\" (UID: \"abd6e8a3-e983-47bb-a937-890763c5eb26\") " pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" Jun 11 21:19:44.799050 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:19:44.798949 2563 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jun 11 21:19:44.799050 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:19:44.799037 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abd6e8a3-e983-47bb-a937-890763c5eb26-tls-certs podName:abd6e8a3-e983-47bb-a937-890763c5eb26 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:45.799004669 +0000 UTC m=+559.542084635 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/abd6e8a3-e983-47bb-a937-890763c5eb26-tls-certs") pod "kube-auth-proxy-6c9dfcd5c9-64xrk" (UID: "abd6e8a3-e983-47bb-a937-890763c5eb26") : secret "kube-auth-proxy-tls" not found Jun 11 21:19:45.807314 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.807280 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abd6e8a3-e983-47bb-a937-890763c5eb26-tls-certs\") pod \"kube-auth-proxy-6c9dfcd5c9-64xrk\" (UID: \"abd6e8a3-e983-47bb-a937-890763c5eb26\") " pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" Jun 11 21:19:45.809607 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.809585 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abd6e8a3-e983-47bb-a937-890763c5eb26-tls-certs\") pod \"kube-auth-proxy-6c9dfcd5c9-64xrk\" (UID: \"abd6e8a3-e983-47bb-a937-890763c5eb26\") " pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" Jun 11 21:19:45.882847 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.882815 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx"] Jun 11 21:19:45.886241 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.886226 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:19:45.889419 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.889396 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 11 21:19:45.889557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.889399 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 11 21:19:45.889557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.889497 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 11 21:19:45.889557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.889497 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 11 21:19:45.889718 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.889626 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vzlq2\"" Jun 11 21:19:45.889718 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.889649 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 11 21:19:45.893213 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.893187 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx"] Jun 11 21:19:45.909110 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.909059 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9674668a-7839-4a01-9bdb-10292c7225c2-manager-config\") pod \"lws-controller-manager-5dffd8664c-k5drx\" (UID: \"9674668a-7839-4a01-9bdb-10292c7225c2\") " pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:19:45.909289 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.909149 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj7h4\" (UniqueName: \"kubernetes.io/projected/9674668a-7839-4a01-9bdb-10292c7225c2-kube-api-access-rj7h4\") pod \"lws-controller-manager-5dffd8664c-k5drx\" (UID: \"9674668a-7839-4a01-9bdb-10292c7225c2\") " pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:19:45.909289 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.909183 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9674668a-7839-4a01-9bdb-10292c7225c2-metrics-cert\") pod \"lws-controller-manager-5dffd8664c-k5drx\" (UID: \"9674668a-7839-4a01-9bdb-10292c7225c2\") " pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:19:45.909289 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.909223 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9674668a-7839-4a01-9bdb-10292c7225c2-cert\") pod \"lws-controller-manager-5dffd8664c-k5drx\" (UID: \"9674668a-7839-4a01-9bdb-10292c7225c2\") " pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:19:45.966160 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:45.966124 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" Jun 11 21:19:46.009648 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:46.009616 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9674668a-7839-4a01-9bdb-10292c7225c2-manager-config\") pod \"lws-controller-manager-5dffd8664c-k5drx\" (UID: \"9674668a-7839-4a01-9bdb-10292c7225c2\") " pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:19:46.009803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:46.009671 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj7h4\" (UniqueName: \"kubernetes.io/projected/9674668a-7839-4a01-9bdb-10292c7225c2-kube-api-access-rj7h4\") pod \"lws-controller-manager-5dffd8664c-k5drx\" (UID: \"9674668a-7839-4a01-9bdb-10292c7225c2\") " pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:19:46.009803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:46.009706 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9674668a-7839-4a01-9bdb-10292c7225c2-metrics-cert\") pod \"lws-controller-manager-5dffd8664c-k5drx\" (UID: \"9674668a-7839-4a01-9bdb-10292c7225c2\") " pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:19:46.009803 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:46.009738 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9674668a-7839-4a01-9bdb-10292c7225c2-cert\") pod \"lws-controller-manager-5dffd8664c-k5drx\" (UID: \"9674668a-7839-4a01-9bdb-10292c7225c2\") " pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:19:46.010419 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:46.010393 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9674668a-7839-4a01-9bdb-10292c7225c2-manager-config\") pod \"lws-controller-manager-5dffd8664c-k5drx\" (UID: \"9674668a-7839-4a01-9bdb-10292c7225c2\") " pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:19:46.012794 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:46.012454 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9674668a-7839-4a01-9bdb-10292c7225c2-cert\") pod \"lws-controller-manager-5dffd8664c-k5drx\" (UID: \"9674668a-7839-4a01-9bdb-10292c7225c2\") " pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:19:46.013414 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:46.012970 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9674668a-7839-4a01-9bdb-10292c7225c2-metrics-cert\") pod \"lws-controller-manager-5dffd8664c-k5drx\" (UID: \"9674668a-7839-4a01-9bdb-10292c7225c2\") " pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:19:46.022718 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:46.022693 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj7h4\" (UniqueName: \"kubernetes.io/projected/9674668a-7839-4a01-9bdb-10292c7225c2-kube-api-access-rj7h4\") pod \"lws-controller-manager-5dffd8664c-k5drx\" (UID: \"9674668a-7839-4a01-9bdb-10292c7225c2\") " pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:19:46.100749 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:46.100722 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk"] Jun 11 21:19:46.103127 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:19:46.103085 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabd6e8a3_e983_47bb_a937_890763c5eb26.slice/crio-f414d7fd94585d82f06c1473b647aa8b1b238fec08a801290ca0636927755a98 WatchSource:0}: Error finding container f414d7fd94585d82f06c1473b647aa8b1b238fec08a801290ca0636927755a98: Status 404 returned error can't find the container with id f414d7fd94585d82f06c1473b647aa8b1b238fec08a801290ca0636927755a98 Jun 11 21:19:46.197215 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:46.197184 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:19:46.320359 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:46.320325 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx"] Jun 11 21:19:46.323976 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:19:46.323953 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9674668a_7839_4a01_9bdb_10292c7225c2.slice/crio-6875ddd9efcb88869a7d5182d5617ccaaaa40637a5b76e6a2b3e32fb80b977a9 WatchSource:0}: Error finding container 6875ddd9efcb88869a7d5182d5617ccaaaa40637a5b76e6a2b3e32fb80b977a9: Status 404 returned error can't find the container with id 6875ddd9efcb88869a7d5182d5617ccaaaa40637a5b76e6a2b3e32fb80b977a9 Jun 11 21:19:46.529976 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:46.529890 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" event={"ID":"abd6e8a3-e983-47bb-a937-890763c5eb26","Type":"ContainerStarted","Data":"f414d7fd94585d82f06c1473b647aa8b1b238fec08a801290ca0636927755a98"} Jun 11 21:19:46.530919 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:46.530889 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" event={"ID":"9674668a-7839-4a01-9bdb-10292c7225c2","Type":"ContainerStarted","Data":"6875ddd9efcb88869a7d5182d5617ccaaaa40637a5b76e6a2b3e32fb80b977a9"} Jun 11 21:19:50.549859 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:50.549807 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" event={"ID":"abd6e8a3-e983-47bb-a937-890763c5eb26","Type":"ContainerStarted","Data":"a8379b7a8c13facba8ea9a5a24b395ddc7412822db7e674585bf0db30414be8d"} Jun 11 21:19:50.569620 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:50.569571 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6c9dfcd5c9-64xrk" podStartSLOduration=2.896545953 podStartE2EDuration="6.569555836s" totalImagesPullingTime="3.673009883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:19:50.567494738 +0000 UTC m=+564.310574729" watchObservedRunningTime="2026-06-11 21:19:50.569555836 +0000 UTC m=+564.312635825" Jun 11 21:19:52.559322 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:52.559280 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" event={"ID":"9674668a-7839-4a01-9bdb-10292c7225c2","Type":"ContainerStarted","Data":"b3cea2a1934582e8b94f6b311c649c1d35864334513e4980769ed2b0f6359de1"} Jun 11 21:19:52.559700 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:52.559384 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:19:52.574249 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:19:52.574195 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" podStartSLOduration=2.175468186 podStartE2EDuration="7.574176913s" totalImagesPullingTime="5.398708727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:19:52.573692633 +0000 UTC m=+566.316772634" watchObservedRunningTime="2026-06-11 21:19:52.574176913 +0000 UTC m=+566.317256899" Jun 11 21:20:03.568649 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:20:03.568611 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5dffd8664c-k5drx" Jun 11 21:20:26.584677 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:20:26.584644 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:20:26.588236 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:20:26.588214 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:21:27.116217 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.116176 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2"] Jun 11 21:21:27.119555 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.119535 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2" Jun 11 21:21:27.122006 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.121983 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 11 21:21:27.122172 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.122016 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 11 21:21:27.122172 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.122092 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 11 21:21:27.122172 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.122093 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 11 21:21:27.122843 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.122822 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6zg5l\"" Jun 11 21:21:27.128328 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.128306 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2"] Jun 11 21:21:27.190130 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.190090 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/525d6a7a-f7d7-445d-a489-9b9f888fce17-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-r4zn2\" (UID: \"525d6a7a-f7d7-445d-a489-9b9f888fce17\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2" Jun 11 21:21:27.190130 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.190139 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/525d6a7a-f7d7-445d-a489-9b9f888fce17-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-r4zn2\" (UID: \"525d6a7a-f7d7-445d-a489-9b9f888fce17\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2" Jun 11 21:21:27.190349 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.190184 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xht9k\" (UniqueName: \"kubernetes.io/projected/525d6a7a-f7d7-445d-a489-9b9f888fce17-kube-api-access-xht9k\") pod \"kuadrant-console-plugin-66b5774985-r4zn2\" (UID: \"525d6a7a-f7d7-445d-a489-9b9f888fce17\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2" Jun 11 21:21:27.290812 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.290778 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/525d6a7a-f7d7-445d-a489-9b9f888fce17-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-r4zn2\" (UID: \"525d6a7a-f7d7-445d-a489-9b9f888fce17\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2" Jun 11 21:21:27.290968 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.290820 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/525d6a7a-f7d7-445d-a489-9b9f888fce17-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-r4zn2\" (UID: \"525d6a7a-f7d7-445d-a489-9b9f888fce17\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2" Jun 11 21:21:27.290968 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.290859 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xht9k\" (UniqueName: \"kubernetes.io/projected/525d6a7a-f7d7-445d-a489-9b9f888fce17-kube-api-access-xht9k\") pod \"kuadrant-console-plugin-66b5774985-r4zn2\" (UID: \"525d6a7a-f7d7-445d-a489-9b9f888fce17\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2" Jun 11 21:21:27.290968 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:21:27.290912 2563 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jun 11 21:21:27.290968 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:21:27.290968 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/525d6a7a-f7d7-445d-a489-9b9f888fce17-plugin-serving-cert podName:525d6a7a-f7d7-445d-a489-9b9f888fce17 nodeName:}" failed. No retries permitted until 2026-06-11 21:21:27.790952835 +0000 UTC m=+661.534032804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/525d6a7a-f7d7-445d-a489-9b9f888fce17-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-r4zn2" (UID: "525d6a7a-f7d7-445d-a489-9b9f888fce17") : secret "plugin-serving-cert" not found Jun 11 21:21:27.291589 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.291567 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/525d6a7a-f7d7-445d-a489-9b9f888fce17-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-r4zn2\" (UID: \"525d6a7a-f7d7-445d-a489-9b9f888fce17\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2" Jun 11 21:21:27.300922 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.300901 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xht9k\" (UniqueName: \"kubernetes.io/projected/525d6a7a-f7d7-445d-a489-9b9f888fce17-kube-api-access-xht9k\") pod \"kuadrant-console-plugin-66b5774985-r4zn2\" (UID: \"525d6a7a-f7d7-445d-a489-9b9f888fce17\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2" Jun 11 21:21:27.795727 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.795684 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/525d6a7a-f7d7-445d-a489-9b9f888fce17-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-r4zn2\" (UID: \"525d6a7a-f7d7-445d-a489-9b9f888fce17\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2" Jun 11 21:21:27.798128 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:27.798108 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/525d6a7a-f7d7-445d-a489-9b9f888fce17-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-r4zn2\" (UID: \"525d6a7a-f7d7-445d-a489-9b9f888fce17\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2" Jun 11 21:21:28.029353 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:28.029321 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2" Jun 11 21:21:28.152940 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:28.152915 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2"] Jun 11 21:21:28.155486 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:21:28.155454 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod525d6a7a_f7d7_445d_a489_9b9f888fce17.slice/crio-c3bbbfd742bbd8dcc7f41fa469069f154c3d59e79e63311c662ba67d0520b4a9 WatchSource:0}: Error finding container c3bbbfd742bbd8dcc7f41fa469069f154c3d59e79e63311c662ba67d0520b4a9: Status 404 returned error can't find the container with id c3bbbfd742bbd8dcc7f41fa469069f154c3d59e79e63311c662ba67d0520b4a9 Jun 11 21:21:28.914353 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:28.914296 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2" event={"ID":"525d6a7a-f7d7-445d-a489-9b9f888fce17","Type":"ContainerStarted","Data":"c3bbbfd742bbd8dcc7f41fa469069f154c3d59e79e63311c662ba67d0520b4a9"} Jun 11 21:21:56.037809 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:56.037773 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2" event={"ID":"525d6a7a-f7d7-445d-a489-9b9f888fce17","Type":"ContainerStarted","Data":"62639f07a126bacc9afae71c12fea5084908a4789f9a9b3e8f9ddb87d3e0726d"} Jun 11 21:21:56.056054 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:21:56.055988 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r4zn2" podStartSLOduration=2.239877202 podStartE2EDuration="29.055975816s" totalImagesPullingTime="26.816098614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:21:56.054806647 +0000 UTC m=+689.797886638" watchObservedRunningTime="2026-06-11 21:21:56.055975816 +0000 UTC m=+689.799055847" Jun 11 21:22:11.087518 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:11.087480 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-22djj"] Jun 11 21:22:11.097457 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:11.097379 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" Jun 11 21:22:11.099668 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:11.099640 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-22djj"] Jun 11 21:22:11.099808 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:11.099689 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 11 21:22:11.185243 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:11.185204 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7m4k\" (UniqueName: \"kubernetes.io/projected/c07b37f0-29bf-4bf2-8353-d0f57433aaaa-kube-api-access-m7m4k\") pod \"limitador-limitador-64c94fd64f-22djj\" (UID: \"c07b37f0-29bf-4bf2-8353-d0f57433aaaa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" Jun 11 21:22:11.185243 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:11.185242 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c07b37f0-29bf-4bf2-8353-d0f57433aaaa-config-file\") pod \"limitador-limitador-64c94fd64f-22djj\" (UID: \"c07b37f0-29bf-4bf2-8353-d0f57433aaaa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" Jun 11 21:22:11.186262 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:11.186239 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-22djj"] Jun 11 21:22:11.286658 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:11.286627 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7m4k\" (UniqueName: \"kubernetes.io/projected/c07b37f0-29bf-4bf2-8353-d0f57433aaaa-kube-api-access-m7m4k\") pod \"limitador-limitador-64c94fd64f-22djj\" (UID: \"c07b37f0-29bf-4bf2-8353-d0f57433aaaa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" Jun 11 21:22:11.286658 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:11.286663 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c07b37f0-29bf-4bf2-8353-d0f57433aaaa-config-file\") pod \"limitador-limitador-64c94fd64f-22djj\" (UID: \"c07b37f0-29bf-4bf2-8353-d0f57433aaaa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" Jun 11 21:22:11.287455 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:11.287424 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c07b37f0-29bf-4bf2-8353-d0f57433aaaa-config-file\") pod \"limitador-limitador-64c94fd64f-22djj\" (UID: \"c07b37f0-29bf-4bf2-8353-d0f57433aaaa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" Jun 11 21:22:11.295579 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:11.295555 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7m4k\" (UniqueName: \"kubernetes.io/projected/c07b37f0-29bf-4bf2-8353-d0f57433aaaa-kube-api-access-m7m4k\") pod \"limitador-limitador-64c94fd64f-22djj\" (UID: \"c07b37f0-29bf-4bf2-8353-d0f57433aaaa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" Jun 11 21:22:11.408604 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:11.408515 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" Jun 11 21:22:11.536082 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:11.535978 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-22djj"] Jun 11 21:22:11.538535 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:22:11.538508 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc07b37f0_29bf_4bf2_8353_d0f57433aaaa.slice/crio-296110dbcde9caef2488b5a4eab590e36504388b5b97ca73c5726bc91af8e9ff WatchSource:0}: Error finding container 296110dbcde9caef2488b5a4eab590e36504388b5b97ca73c5726bc91af8e9ff: Status 404 returned error can't find the container with id 296110dbcde9caef2488b5a4eab590e36504388b5b97ca73c5726bc91af8e9ff Jun 11 21:22:12.084893 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:12.084861 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-hz8z5"] Jun 11 21:22:12.087857 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:12.087838 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hz8z5" Jun 11 21:22:12.090327 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:12.090308 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8bp4b\"" Jun 11 21:22:12.097120 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:12.097092 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hz8z5"] Jun 11 21:22:12.105329 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:12.105304 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" event={"ID":"c07b37f0-29bf-4bf2-8353-d0f57433aaaa","Type":"ContainerStarted","Data":"296110dbcde9caef2488b5a4eab590e36504388b5b97ca73c5726bc91af8e9ff"} Jun 11 21:22:12.194490 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:12.194456 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwlbd\" (UniqueName: \"kubernetes.io/projected/d11d63e8-eb2a-4445-8c90-f95b01d80645-kube-api-access-rwlbd\") pod \"authorino-75d556bd77-hz8z5\" (UID: \"d11d63e8-eb2a-4445-8c90-f95b01d80645\") " pod="kuadrant-system/authorino-75d556bd77-hz8z5" Jun 11 21:22:12.295995 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:12.295961 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwlbd\" (UniqueName: \"kubernetes.io/projected/d11d63e8-eb2a-4445-8c90-f95b01d80645-kube-api-access-rwlbd\") pod \"authorino-75d556bd77-hz8z5\" (UID: \"d11d63e8-eb2a-4445-8c90-f95b01d80645\") " pod="kuadrant-system/authorino-75d556bd77-hz8z5" Jun 11 21:22:12.307072 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:12.307046 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwlbd\" (UniqueName: \"kubernetes.io/projected/d11d63e8-eb2a-4445-8c90-f95b01d80645-kube-api-access-rwlbd\") pod \"authorino-75d556bd77-hz8z5\" (UID: \"d11d63e8-eb2a-4445-8c90-f95b01d80645\") " pod="kuadrant-system/authorino-75d556bd77-hz8z5" Jun 11 21:22:12.397178 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:12.397098 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hz8z5" Jun 11 21:22:12.571862 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:12.570138 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hz8z5"] Jun 11 21:22:12.573304 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:22:12.573270 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd11d63e8_eb2a_4445_8c90_f95b01d80645.slice/crio-b4a7fb90b2c47a7b2a9dbc2c519afc7c56fc5a03155501e8a670e0d72e9a1a50 WatchSource:0}: Error finding container b4a7fb90b2c47a7b2a9dbc2c519afc7c56fc5a03155501e8a670e0d72e9a1a50: Status 404 returned error can't find the container with id b4a7fb90b2c47a7b2a9dbc2c519afc7c56fc5a03155501e8a670e0d72e9a1a50 Jun 11 21:22:13.118547 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:13.118503 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hz8z5" event={"ID":"d11d63e8-eb2a-4445-8c90-f95b01d80645","Type":"ContainerStarted","Data":"b4a7fb90b2c47a7b2a9dbc2c519afc7c56fc5a03155501e8a670e0d72e9a1a50"} Jun 11 21:22:17.140693 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:17.140655 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hz8z5" event={"ID":"d11d63e8-eb2a-4445-8c90-f95b01d80645","Type":"ContainerStarted","Data":"6e19754bfc1dee4d664a102c2689134e9138416d318b7b0c48e6e025e9ebdd1b"} Jun 11 21:22:17.142395 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:17.142366 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" event={"ID":"c07b37f0-29bf-4bf2-8353-d0f57433aaaa","Type":"ContainerStarted","Data":"62185ccb803a1ae1a69b3b78818931f6495537791bc4acbcf0c4e33c034f933d"} Jun 11 21:22:17.142538 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:17.142511 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" Jun 11 21:22:17.155370 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:17.155324 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-hz8z5" podStartSLOduration=1.313215896 podStartE2EDuration="5.155310773s" totalImagesPullingTime="3.842094877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:22:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:22:17.153792726 +0000 UTC m=+710.896872730" watchObservedRunningTime="2026-06-11 21:22:17.155310773 +0000 UTC m=+710.898390761" Jun 11 21:22:26.124938 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:26.124880 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" podStartSLOduration=10.254238101 podStartE2EDuration="15.124862693s" totalImagesPullingTime="4.870624592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:22:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:22:17.169271192 +0000 UTC m=+710.912351184" watchObservedRunningTime="2026-06-11 21:22:26.124862693 +0000 UTC m=+719.867942729" Jun 11 21:22:26.125458 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:26.125447 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-22djj"] Jun 11 21:22:26.125620 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:26.125600 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" podUID="c07b37f0-29bf-4bf2-8353-d0f57433aaaa" containerName="limitador" containerID="cri-o://62185ccb803a1ae1a69b3b78818931f6495537791bc4acbcf0c4e33c034f933d" gracePeriod=30 Jun 11 21:22:26.126288 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:26.126224 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" Jun 11 21:22:26.724536 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:26.724513 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" Jun 11 21:22:26.826103 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:26.826072 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7m4k\" (UniqueName: \"kubernetes.io/projected/c07b37f0-29bf-4bf2-8353-d0f57433aaaa-kube-api-access-m7m4k\") pod \"c07b37f0-29bf-4bf2-8353-d0f57433aaaa\" (UID: \"c07b37f0-29bf-4bf2-8353-d0f57433aaaa\") " Jun 11 21:22:26.826285 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:26.826127 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c07b37f0-29bf-4bf2-8353-d0f57433aaaa-config-file\") pod \"c07b37f0-29bf-4bf2-8353-d0f57433aaaa\" (UID: \"c07b37f0-29bf-4bf2-8353-d0f57433aaaa\") " Jun 11 21:22:26.826516 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:26.826490 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c07b37f0-29bf-4bf2-8353-d0f57433aaaa-config-file" (OuterVolumeSpecName: "config-file") pod "c07b37f0-29bf-4bf2-8353-d0f57433aaaa" (UID: "c07b37f0-29bf-4bf2-8353-d0f57433aaaa"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:22:26.828190 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:26.828166 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c07b37f0-29bf-4bf2-8353-d0f57433aaaa-kube-api-access-m7m4k" (OuterVolumeSpecName: "kube-api-access-m7m4k") pod "c07b37f0-29bf-4bf2-8353-d0f57433aaaa" (UID: "c07b37f0-29bf-4bf2-8353-d0f57433aaaa"). InnerVolumeSpecName "kube-api-access-m7m4k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:22:26.927534 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:26.927492 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7m4k\" (UniqueName: \"kubernetes.io/projected/c07b37f0-29bf-4bf2-8353-d0f57433aaaa-kube-api-access-m7m4k\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:22:26.927534 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:26.927526 2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c07b37f0-29bf-4bf2-8353-d0f57433aaaa-config-file\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:22:27.183467 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.183436 2563 generic.go:356] "Generic (PLEG): container finished" podID="c07b37f0-29bf-4bf2-8353-d0f57433aaaa" containerID="62185ccb803a1ae1a69b3b78818931f6495537791bc4acbcf0c4e33c034f933d" exitCode=0 Jun 11 21:22:27.183899 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.183476 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" event={"ID":"c07b37f0-29bf-4bf2-8353-d0f57433aaaa","Type":"ContainerDied","Data":"62185ccb803a1ae1a69b3b78818931f6495537791bc4acbcf0c4e33c034f933d"} Jun 11 21:22:27.183899 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.183481 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" Jun 11 21:22:27.183899 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.183513 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-22djj" event={"ID":"c07b37f0-29bf-4bf2-8353-d0f57433aaaa","Type":"ContainerDied","Data":"296110dbcde9caef2488b5a4eab590e36504388b5b97ca73c5726bc91af8e9ff"} Jun 11 21:22:27.183899 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.183534 2563 scope.go:117] "RemoveContainer" containerID="62185ccb803a1ae1a69b3b78818931f6495537791bc4acbcf0c4e33c034f933d" Jun 11 21:22:27.192444 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.192426 2563 scope.go:117] "RemoveContainer" containerID="62185ccb803a1ae1a69b3b78818931f6495537791bc4acbcf0c4e33c034f933d" Jun 11 21:22:27.192680 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:22:27.192660 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62185ccb803a1ae1a69b3b78818931f6495537791bc4acbcf0c4e33c034f933d\": container with ID starting with 62185ccb803a1ae1a69b3b78818931f6495537791bc4acbcf0c4e33c034f933d not found: ID does not exist" containerID="62185ccb803a1ae1a69b3b78818931f6495537791bc4acbcf0c4e33c034f933d" Jun 11 21:22:27.192725 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.192690 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62185ccb803a1ae1a69b3b78818931f6495537791bc4acbcf0c4e33c034f933d"} err="failed to get container status \"62185ccb803a1ae1a69b3b78818931f6495537791bc4acbcf0c4e33c034f933d\": rpc error: code = NotFound desc = could not find container \"62185ccb803a1ae1a69b3b78818931f6495537791bc4acbcf0c4e33c034f933d\": container with ID starting with 62185ccb803a1ae1a69b3b78818931f6495537791bc4acbcf0c4e33c034f933d not found: ID does not exist" Jun 11 21:22:27.206167 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.206141 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-22djj"] Jun 11 21:22:27.211731 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.211711 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-22djj"] Jun 11 21:22:27.223787 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.223764 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-jrp4h"] Jun 11 21:22:27.224069 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.224056 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c07b37f0-29bf-4bf2-8353-d0f57433aaaa" containerName="limitador" Jun 11 21:22:27.224125 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.224070 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c07b37f0-29bf-4bf2-8353-d0f57433aaaa" containerName="limitador" Jun 11 21:22:27.224163 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.224140 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c07b37f0-29bf-4bf2-8353-d0f57433aaaa" containerName="limitador" Jun 11 21:22:27.228425 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.228408 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-jrp4h" Jun 11 21:22:27.230481 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.230456 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-t9zwd\"" Jun 11 21:22:27.230481 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.230467 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 11 21:22:27.234487 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.234428 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-jrp4h"] Jun 11 21:22:27.331146 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.331106 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mx8b\" (UniqueName: \"kubernetes.io/projected/f3b07c49-faaf-49ea-bd22-23102814a7b2-kube-api-access-5mx8b\") pod \"postgres-69fb4594cf-jrp4h\" (UID: \"f3b07c49-faaf-49ea-bd22-23102814a7b2\") " pod="opendatahub/postgres-69fb4594cf-jrp4h" Jun 11 21:22:27.331341 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.331207 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f3b07c49-faaf-49ea-bd22-23102814a7b2-data\") pod \"postgres-69fb4594cf-jrp4h\" (UID: \"f3b07c49-faaf-49ea-bd22-23102814a7b2\") " pod="opendatahub/postgres-69fb4594cf-jrp4h" Jun 11 21:22:27.432513 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.432477 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f3b07c49-faaf-49ea-bd22-23102814a7b2-data\") pod \"postgres-69fb4594cf-jrp4h\" (UID: \"f3b07c49-faaf-49ea-bd22-23102814a7b2\") " pod="opendatahub/postgres-69fb4594cf-jrp4h" Jun 11 21:22:27.432706 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.432564 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mx8b\" (UniqueName: \"kubernetes.io/projected/f3b07c49-faaf-49ea-bd22-23102814a7b2-kube-api-access-5mx8b\") pod \"postgres-69fb4594cf-jrp4h\" (UID: \"f3b07c49-faaf-49ea-bd22-23102814a7b2\") " pod="opendatahub/postgres-69fb4594cf-jrp4h" Jun 11 21:22:27.432910 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.432887 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f3b07c49-faaf-49ea-bd22-23102814a7b2-data\") pod \"postgres-69fb4594cf-jrp4h\" (UID: \"f3b07c49-faaf-49ea-bd22-23102814a7b2\") " pod="opendatahub/postgres-69fb4594cf-jrp4h" Jun 11 21:22:27.440640 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.440589 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mx8b\" (UniqueName: \"kubernetes.io/projected/f3b07c49-faaf-49ea-bd22-23102814a7b2-kube-api-access-5mx8b\") pod \"postgres-69fb4594cf-jrp4h\" (UID: \"f3b07c49-faaf-49ea-bd22-23102814a7b2\") " pod="opendatahub/postgres-69fb4594cf-jrp4h" Jun 11 21:22:27.539766 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.539732 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-jrp4h" Jun 11 21:22:27.663898 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:27.663865 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-jrp4h"] Jun 11 21:22:27.666786 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:22:27.666756 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3b07c49_faaf_49ea_bd22_23102814a7b2.slice/crio-a1ad6b52b2b3656a293d4f265c14c2d9d2071676c88a636e9c47838cd1f1a670 WatchSource:0}: Error finding container a1ad6b52b2b3656a293d4f265c14c2d9d2071676c88a636e9c47838cd1f1a670: Status 404 returned error can't find the container with id a1ad6b52b2b3656a293d4f265c14c2d9d2071676c88a636e9c47838cd1f1a670 Jun 11 21:22:28.189587 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:28.189553 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-jrp4h" event={"ID":"f3b07c49-faaf-49ea-bd22-23102814a7b2","Type":"ContainerStarted","Data":"a1ad6b52b2b3656a293d4f265c14c2d9d2071676c88a636e9c47838cd1f1a670"} Jun 11 21:22:28.616930 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:28.616889 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c07b37f0-29bf-4bf2-8353-d0f57433aaaa" path="/var/lib/kubelet/pods/c07b37f0-29bf-4bf2-8353-d0f57433aaaa/volumes" Jun 11 21:22:31.205346 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:31.205307 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-jrp4h" event={"ID":"f3b07c49-faaf-49ea-bd22-23102814a7b2","Type":"ContainerStarted","Data":"31c6ee8ec0b3b47feac3ad6fcf0ce4698232611b3502629d05bb925562f7cc27"} Jun 11 21:22:31.205749 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:31.205402 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-jrp4h" Jun 11 21:22:31.220723 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:31.220673 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-jrp4h" podStartSLOduration=1.56113402 podStartE2EDuration="4.220658447s" totalImagesPullingTime="2.659524427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:22:31.219880163 +0000 UTC m=+724.962960152" watchObservedRunningTime="2026-06-11 21:22:31.220658447 +0000 UTC m=+724.963738435" Jun 11 21:22:37.245425 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:22:37.245394 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-jrp4h" Jun 11 21:23:01.586527 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:01.586494 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 21:23:01.598043 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:01.597990 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 21:23:01.600113 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:01.600082 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 21:23:01.600250 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:01.600225 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 11 21:23:01.600325 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:01.600281 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-b4v4x\"" Jun 11 21:23:01.600325 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:01.600225 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 11 21:23:01.600325 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:01.600304 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 11 21:23:01.717761 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:01.717727 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkqmk\" (UniqueName: \"kubernetes.io/projected/914d36e1-ea89-4cd1-afd9-905bbf589d92-kube-api-access-hkqmk\") pod \"maas-keycloak-0\" (UID: \"914d36e1-ea89-4cd1-afd9-905bbf589d92\") " pod="keycloak-system/maas-keycloak-0" Jun 11 21:23:01.818792 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:01.818758 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkqmk\" (UniqueName: \"kubernetes.io/projected/914d36e1-ea89-4cd1-afd9-905bbf589d92-kube-api-access-hkqmk\") pod \"maas-keycloak-0\" (UID: \"914d36e1-ea89-4cd1-afd9-905bbf589d92\") " pod="keycloak-system/maas-keycloak-0" Jun 11 21:23:01.827695 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:01.827661 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkqmk\" (UniqueName: \"kubernetes.io/projected/914d36e1-ea89-4cd1-afd9-905bbf589d92-kube-api-access-hkqmk\") pod \"maas-keycloak-0\" (UID: \"914d36e1-ea89-4cd1-afd9-905bbf589d92\") " pod="keycloak-system/maas-keycloak-0" Jun 11 21:23:01.908159 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:01.908068 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 21:23:02.037521 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:02.037496 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 21:23:02.040038 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:23:02.039993 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod914d36e1_ea89_4cd1_afd9_905bbf589d92.slice/crio-fab4b889305e3fd66986c83eb0b21f83aae25a688317ab688f327addb3e7f32a WatchSource:0}: Error finding container fab4b889305e3fd66986c83eb0b21f83aae25a688317ab688f327addb3e7f32a: Status 404 returned error can't find the container with id fab4b889305e3fd66986c83eb0b21f83aae25a688317ab688f327addb3e7f32a Jun 11 21:23:02.333271 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:02.333237 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"914d36e1-ea89-4cd1-afd9-905bbf589d92","Type":"ContainerStarted","Data":"fab4b889305e3fd66986c83eb0b21f83aae25a688317ab688f327addb3e7f32a"} Jun 11 21:23:07.358981 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:07.358941 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"914d36e1-ea89-4cd1-afd9-905bbf589d92","Type":"ContainerStarted","Data":"f125e01f800682447b5cc300bac8ce07f5e513f7b290dc25bc480d3239826c3c"} Jun 11 21:23:07.376392 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:07.376329 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.873236979 podStartE2EDuration="6.376307983s" totalImagesPullingTime="4.503071004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:23:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:23:07.37424175 +0000 UTC m=+761.117321753" watchObservedRunningTime="2026-06-11 21:23:07.376307983 +0000 UTC m=+761.119387974" Jun 11 21:23:07.908783 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:07.908728 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 11 21:23:07.910802 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:07.910760 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.25:9000/health/started\": dial tcp 10.133.0.25:9000: connect: connection refused" Jun 11 21:23:08.909240 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:08.909177 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.25:9000/health/started\": dial tcp 10.133.0.25:9000: connect: connection refused" Jun 11 21:23:09.909014 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:09.908960 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.25:9000/health/started\": dial tcp 10.133.0.25:9000: connect: connection refused" Jun 11 21:23:10.908761 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:10.908687 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.25:9000/health/started\": dial tcp 10.133.0.25:9000: connect: connection refused" Jun 11 21:23:11.908273 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:11.908228 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 11 21:23:11.908623 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:11.908572 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.25:9000/health/started\": dial tcp 10.133.0.25:9000: connect: connection refused" Jun 11 21:23:12.909067 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:12.908998 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.25:9000/health/started\": dial tcp 10.133.0.25:9000: connect: connection refused" Jun 11 21:23:13.910001 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:13.909547 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.25:9000/health/started\": dial tcp 10.133.0.25:9000: connect: connection refused" Jun 11 21:23:14.909530 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:14.909481 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.25:9000/health/started\": dial tcp 10.133.0.25:9000: connect: connection refused" Jun 11 21:23:15.909271 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:15.909220 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.25:9000/health/started\": dial tcp 10.133.0.25:9000: connect: connection refused" Jun 11 21:23:16.908907 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:16.908850 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.25:9000/health/started\": dial tcp 10.133.0.25:9000: connect: connection refused" Jun 11 21:23:17.909395 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:17.909344 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.25:9000/health/started\": dial tcp 10.133.0.25:9000: connect: connection refused" Jun 11 21:23:18.909529 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:18.909470 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.25:9000/health/started\": dial tcp 10.133.0.25:9000: connect: connection refused" Jun 11 21:23:20.020558 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:20.020492 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 11 21:23:20.039589 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:20.039550 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 21:23:30.027363 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:30.027326 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 11 21:23:33.299562 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:33.299526 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hz8z5"] Jun 11 21:23:33.300074 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:33.299704 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-hz8z5" podUID="d11d63e8-eb2a-4445-8c90-f95b01d80645" containerName="authorino" containerID="cri-o://6e19754bfc1dee4d664a102c2689134e9138416d318b7b0c48e6e025e9ebdd1b" gracePeriod=30 Jun 11 21:23:33.496970 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:33.496941 2563 generic.go:356] "Generic (PLEG): container finished" podID="d11d63e8-eb2a-4445-8c90-f95b01d80645" containerID="6e19754bfc1dee4d664a102c2689134e9138416d318b7b0c48e6e025e9ebdd1b" exitCode=0 Jun 11 21:23:33.497120 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:33.497042 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hz8z5" event={"ID":"d11d63e8-eb2a-4445-8c90-f95b01d80645","Type":"ContainerDied","Data":"6e19754bfc1dee4d664a102c2689134e9138416d318b7b0c48e6e025e9ebdd1b"} Jun 11 21:23:33.566691 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:33.566659 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hz8z5" Jun 11 21:23:33.600126 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:33.600091 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwlbd\" (UniqueName: \"kubernetes.io/projected/d11d63e8-eb2a-4445-8c90-f95b01d80645-kube-api-access-rwlbd\") pod \"d11d63e8-eb2a-4445-8c90-f95b01d80645\" (UID: \"d11d63e8-eb2a-4445-8c90-f95b01d80645\") " Jun 11 21:23:33.602358 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:33.602326 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d11d63e8-eb2a-4445-8c90-f95b01d80645-kube-api-access-rwlbd" (OuterVolumeSpecName: "kube-api-access-rwlbd") pod "d11d63e8-eb2a-4445-8c90-f95b01d80645" (UID: "d11d63e8-eb2a-4445-8c90-f95b01d80645"). InnerVolumeSpecName "kube-api-access-rwlbd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:23:33.701099 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:33.701061 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwlbd\" (UniqueName: \"kubernetes.io/projected/d11d63e8-eb2a-4445-8c90-f95b01d80645-kube-api-access-rwlbd\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:23:34.502865 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:34.502831 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hz8z5" Jun 11 21:23:34.503327 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:34.502837 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hz8z5" event={"ID":"d11d63e8-eb2a-4445-8c90-f95b01d80645","Type":"ContainerDied","Data":"b4a7fb90b2c47a7b2a9dbc2c519afc7c56fc5a03155501e8a670e0d72e9a1a50"} Jun 11 21:23:34.503327 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:34.502962 2563 scope.go:117] "RemoveContainer" containerID="6e19754bfc1dee4d664a102c2689134e9138416d318b7b0c48e6e025e9ebdd1b" Jun 11 21:23:34.522775 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:34.522744 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hz8z5"] Jun 11 21:23:34.529122 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:34.529099 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hz8z5"] Jun 11 21:23:34.617121 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:34.617089 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d11d63e8-eb2a-4445-8c90-f95b01d80645" path="/var/lib/kubelet/pods/d11d63e8-eb2a-4445-8c90-f95b01d80645/volumes" Jun 11 21:23:36.875828 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:36.875794 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-584789bccc-4gppn"] Jun 11 21:23:36.876215 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:36.876105 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d11d63e8-eb2a-4445-8c90-f95b01d80645" containerName="authorino" Jun 11 21:23:36.876215 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:36.876117 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d11d63e8-eb2a-4445-8c90-f95b01d80645" containerName="authorino" Jun 11 21:23:36.876215 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:36.876167 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d11d63e8-eb2a-4445-8c90-f95b01d80645" containerName="authorino" Jun 11 21:23:36.879825 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:36.879808 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-584789bccc-4gppn" Jun 11 21:23:36.882461 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:36.882437 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 11 21:23:36.882612 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:36.882503 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 11 21:23:36.882612 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:36.882521 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-slhdv\"" Jun 11 21:23:36.889217 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:36.889192 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-584789bccc-4gppn"] Jun 11 21:23:36.931112 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:36.931081 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4jrt\" (UniqueName: \"kubernetes.io/projected/a168313a-978a-4f8c-b3e7-4e00816ed9cc-kube-api-access-q4jrt\") pod \"maas-controller-584789bccc-4gppn\" (UID: \"a168313a-978a-4f8c-b3e7-4e00816ed9cc\") " pod="opendatahub/maas-controller-584789bccc-4gppn" Jun 11 21:23:36.931112 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:36.931117 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a168313a-978a-4f8c-b3e7-4e00816ed9cc-webhook-cert\") pod \"maas-controller-584789bccc-4gppn\" (UID: \"a168313a-978a-4f8c-b3e7-4e00816ed9cc\") " pod="opendatahub/maas-controller-584789bccc-4gppn" Jun 11 21:23:37.032245 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.032209 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4jrt\" (UniqueName: \"kubernetes.io/projected/a168313a-978a-4f8c-b3e7-4e00816ed9cc-kube-api-access-q4jrt\") pod \"maas-controller-584789bccc-4gppn\" (UID: \"a168313a-978a-4f8c-b3e7-4e00816ed9cc\") " pod="opendatahub/maas-controller-584789bccc-4gppn" Jun 11 21:23:37.032245 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.032246 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a168313a-978a-4f8c-b3e7-4e00816ed9cc-webhook-cert\") pod \"maas-controller-584789bccc-4gppn\" (UID: \"a168313a-978a-4f8c-b3e7-4e00816ed9cc\") " pod="opendatahub/maas-controller-584789bccc-4gppn" Jun 11 21:23:37.032530 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:23:37.032373 2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 11 21:23:37.032530 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:23:37.032449 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a168313a-978a-4f8c-b3e7-4e00816ed9cc-webhook-cert podName:a168313a-978a-4f8c-b3e7-4e00816ed9cc nodeName:}" failed. No retries permitted until 2026-06-11 21:23:37.532434341 +0000 UTC m=+791.275514306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/a168313a-978a-4f8c-b3e7-4e00816ed9cc-webhook-cert") pod "maas-controller-584789bccc-4gppn" (UID: "a168313a-978a-4f8c-b3e7-4e00816ed9cc") : secret "maas-controller-webhook-cert" not found Jun 11 21:23:37.045209 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.045174 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4jrt\" (UniqueName: \"kubernetes.io/projected/a168313a-978a-4f8c-b3e7-4e00816ed9cc-kube-api-access-q4jrt\") pod \"maas-controller-584789bccc-4gppn\" (UID: \"a168313a-978a-4f8c-b3e7-4e00816ed9cc\") " pod="opendatahub/maas-controller-584789bccc-4gppn" Jun 11 21:23:37.057755 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.057730 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74864b9f6-kvtjf"] Jun 11 21:23:37.060543 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.060529 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74864b9f6-kvtjf" Jun 11 21:23:37.073738 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.073718 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74864b9f6-kvtjf"] Jun 11 21:23:37.132999 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.132911 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4rlj\" (UniqueName: \"kubernetes.io/projected/91dac085-89c5-40d5-a981-a4fd0b03f449-kube-api-access-t4rlj\") pod \"maas-controller-74864b9f6-kvtjf\" (UID: \"91dac085-89c5-40d5-a981-a4fd0b03f449\") " pod="opendatahub/maas-controller-74864b9f6-kvtjf" Jun 11 21:23:37.133182 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.133069 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/91dac085-89c5-40d5-a981-a4fd0b03f449-webhook-cert\") pod \"maas-controller-74864b9f6-kvtjf\" (UID: \"91dac085-89c5-40d5-a981-a4fd0b03f449\") " pod="opendatahub/maas-controller-74864b9f6-kvtjf" Jun 11 21:23:37.234342 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.234310 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4rlj\" (UniqueName: \"kubernetes.io/projected/91dac085-89c5-40d5-a981-a4fd0b03f449-kube-api-access-t4rlj\") pod \"maas-controller-74864b9f6-kvtjf\" (UID: \"91dac085-89c5-40d5-a981-a4fd0b03f449\") " pod="opendatahub/maas-controller-74864b9f6-kvtjf" Jun 11 21:23:37.234492 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.234393 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/91dac085-89c5-40d5-a981-a4fd0b03f449-webhook-cert\") pod \"maas-controller-74864b9f6-kvtjf\" (UID: \"91dac085-89c5-40d5-a981-a4fd0b03f449\") " pod="opendatahub/maas-controller-74864b9f6-kvtjf" Jun 11 21:23:37.236846 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.236823 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/91dac085-89c5-40d5-a981-a4fd0b03f449-webhook-cert\") pod \"maas-controller-74864b9f6-kvtjf\" (UID: \"91dac085-89c5-40d5-a981-a4fd0b03f449\") " pod="opendatahub/maas-controller-74864b9f6-kvtjf" Jun 11 21:23:37.242078 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.242055 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4rlj\" (UniqueName: \"kubernetes.io/projected/91dac085-89c5-40d5-a981-a4fd0b03f449-kube-api-access-t4rlj\") pod \"maas-controller-74864b9f6-kvtjf\" (UID: \"91dac085-89c5-40d5-a981-a4fd0b03f449\") " pod="opendatahub/maas-controller-74864b9f6-kvtjf" Jun 11 21:23:37.370001 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.369967 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74864b9f6-kvtjf" Jun 11 21:23:37.494707 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.494683 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74864b9f6-kvtjf"] Jun 11 21:23:37.497216 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:23:37.497192 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91dac085_89c5_40d5_a981_a4fd0b03f449.slice/crio-13117089d2636494adafd3d040794f00f09c0d8fa624f45e58e4e49edad84110 WatchSource:0}: Error finding container 13117089d2636494adafd3d040794f00f09c0d8fa624f45e58e4e49edad84110: Status 404 returned error can't find the container with id 13117089d2636494adafd3d040794f00f09c0d8fa624f45e58e4e49edad84110 Jun 11 21:23:37.517525 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.517500 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74864b9f6-kvtjf" event={"ID":"91dac085-89c5-40d5-a981-a4fd0b03f449","Type":"ContainerStarted","Data":"13117089d2636494adafd3d040794f00f09c0d8fa624f45e58e4e49edad84110"} Jun 11 21:23:37.537543 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.537515 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a168313a-978a-4f8c-b3e7-4e00816ed9cc-webhook-cert\") pod \"maas-controller-584789bccc-4gppn\" (UID: \"a168313a-978a-4f8c-b3e7-4e00816ed9cc\") " pod="opendatahub/maas-controller-584789bccc-4gppn" Jun 11 21:23:37.539820 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.539802 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a168313a-978a-4f8c-b3e7-4e00816ed9cc-webhook-cert\") pod \"maas-controller-584789bccc-4gppn\" (UID: \"a168313a-978a-4f8c-b3e7-4e00816ed9cc\") " pod="opendatahub/maas-controller-584789bccc-4gppn" Jun 11 21:23:37.791148 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.791059 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-584789bccc-4gppn" Jun 11 21:23:37.917926 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:37.917902 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-584789bccc-4gppn"] Jun 11 21:23:37.920289 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:23:37.920259 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda168313a_978a_4f8c_b3e7_4e00816ed9cc.slice/crio-894da765a80de66e22b6c57bb132a08cbf5477d7391f8f376ecb489fc6e66f75 WatchSource:0}: Error finding container 894da765a80de66e22b6c57bb132a08cbf5477d7391f8f376ecb489fc6e66f75: Status 404 returned error can't find the container with id 894da765a80de66e22b6c57bb132a08cbf5477d7391f8f376ecb489fc6e66f75 Jun 11 21:23:38.525674 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:38.525632 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-584789bccc-4gppn" event={"ID":"a168313a-978a-4f8c-b3e7-4e00816ed9cc","Type":"ContainerStarted","Data":"894da765a80de66e22b6c57bb132a08cbf5477d7391f8f376ecb489fc6e66f75"} Jun 11 21:23:40.535128 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:40.535088 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-584789bccc-4gppn" event={"ID":"a168313a-978a-4f8c-b3e7-4e00816ed9cc","Type":"ContainerStarted","Data":"2299eb90b4070901ed736bd7a28b5aba567487e6938cac848456dabbf05b99b7"} Jun 11 21:23:40.535588 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:40.535172 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-584789bccc-4gppn" Jun 11 21:23:40.536541 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:40.536517 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74864b9f6-kvtjf" event={"ID":"91dac085-89c5-40d5-a981-a4fd0b03f449","Type":"ContainerStarted","Data":"5011cd184b1a3b7e90d418f8dbc3c504ed028e8c1353bfacea16f7109f091d21"} Jun 11 21:23:40.536640 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:40.536566 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74864b9f6-kvtjf" Jun 11 21:23:40.585339 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:40.585287 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-584789bccc-4gppn" podStartSLOduration=2.884034298 podStartE2EDuration="4.585267811s" totalImagesPullingTime="1.701233513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:23:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:23:40.584418812 +0000 UTC m=+794.327498801" watchObservedRunningTime="2026-06-11 21:23:40.585267811 +0000 UTC m=+794.328347800" Jun 11 21:23:40.613905 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:40.613837 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74864b9f6-kvtjf" podStartSLOduration=1.485827053 podStartE2EDuration="3.613817702s" totalImagesPullingTime="2.127990649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:23:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:23:40.611710173 +0000 UTC m=+794.354790192" watchObservedRunningTime="2026-06-11 21:23:40.613817702 +0000 UTC m=+794.356897691" Jun 11 21:23:51.545520 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:51.545490 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74864b9f6-kvtjf" Jun 11 21:23:51.546012 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:51.545643 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-584789bccc-4gppn" Jun 11 21:23:51.603836 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:51.603801 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-584789bccc-4gppn"] Jun 11 21:23:51.604119 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:51.604093 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-584789bccc-4gppn" podUID="a168313a-978a-4f8c-b3e7-4e00816ed9cc" containerName="manager" containerID="cri-o://2299eb90b4070901ed736bd7a28b5aba567487e6938cac848456dabbf05b99b7" gracePeriod=30 Jun 11 21:23:51.850473 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:51.850446 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-584789bccc-4gppn" Jun 11 21:23:51.965254 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:51.965221 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4jrt\" (UniqueName: \"kubernetes.io/projected/a168313a-978a-4f8c-b3e7-4e00816ed9cc-kube-api-access-q4jrt\") pod \"a168313a-978a-4f8c-b3e7-4e00816ed9cc\" (UID: \"a168313a-978a-4f8c-b3e7-4e00816ed9cc\") " Jun 11 21:23:51.965415 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:51.965279 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a168313a-978a-4f8c-b3e7-4e00816ed9cc-webhook-cert\") pod \"a168313a-978a-4f8c-b3e7-4e00816ed9cc\" (UID: \"a168313a-978a-4f8c-b3e7-4e00816ed9cc\") " Jun 11 21:23:51.967488 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:51.967459 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a168313a-978a-4f8c-b3e7-4e00816ed9cc-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a168313a-978a-4f8c-b3e7-4e00816ed9cc" (UID: "a168313a-978a-4f8c-b3e7-4e00816ed9cc"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:23:51.967613 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:51.967492 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a168313a-978a-4f8c-b3e7-4e00816ed9cc-kube-api-access-q4jrt" (OuterVolumeSpecName: "kube-api-access-q4jrt") pod "a168313a-978a-4f8c-b3e7-4e00816ed9cc" (UID: "a168313a-978a-4f8c-b3e7-4e00816ed9cc"). InnerVolumeSpecName "kube-api-access-q4jrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:23:52.066706 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:52.066676 2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a168313a-978a-4f8c-b3e7-4e00816ed9cc-webhook-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:23:52.066706 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:52.066703 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4jrt\" (UniqueName: \"kubernetes.io/projected/a168313a-978a-4f8c-b3e7-4e00816ed9cc-kube-api-access-q4jrt\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:23:52.587960 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:52.587922 2563 generic.go:356] "Generic (PLEG): container finished" podID="a168313a-978a-4f8c-b3e7-4e00816ed9cc" containerID="2299eb90b4070901ed736bd7a28b5aba567487e6938cac848456dabbf05b99b7" exitCode=0 Jun 11 21:23:52.588375 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:52.587966 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-584789bccc-4gppn" Jun 11 21:23:52.588375 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:52.588000 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-584789bccc-4gppn" event={"ID":"a168313a-978a-4f8c-b3e7-4e00816ed9cc","Type":"ContainerDied","Data":"2299eb90b4070901ed736bd7a28b5aba567487e6938cac848456dabbf05b99b7"} Jun 11 21:23:52.588375 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:52.588067 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-584789bccc-4gppn" event={"ID":"a168313a-978a-4f8c-b3e7-4e00816ed9cc","Type":"ContainerDied","Data":"894da765a80de66e22b6c57bb132a08cbf5477d7391f8f376ecb489fc6e66f75"} Jun 11 21:23:52.588375 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:52.588090 2563 scope.go:117] "RemoveContainer" containerID="2299eb90b4070901ed736bd7a28b5aba567487e6938cac848456dabbf05b99b7" Jun 11 21:23:52.596569 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:52.596552 2563 scope.go:117] "RemoveContainer" containerID="2299eb90b4070901ed736bd7a28b5aba567487e6938cac848456dabbf05b99b7" Jun 11 21:23:52.596836 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:23:52.596815 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2299eb90b4070901ed736bd7a28b5aba567487e6938cac848456dabbf05b99b7\": container with ID starting with 2299eb90b4070901ed736bd7a28b5aba567487e6938cac848456dabbf05b99b7 not found: ID does not exist" containerID="2299eb90b4070901ed736bd7a28b5aba567487e6938cac848456dabbf05b99b7" Jun 11 21:23:52.596888 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:52.596844 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2299eb90b4070901ed736bd7a28b5aba567487e6938cac848456dabbf05b99b7"} err="failed to get container status \"2299eb90b4070901ed736bd7a28b5aba567487e6938cac848456dabbf05b99b7\": rpc error: code = NotFound desc = could not find container \"2299eb90b4070901ed736bd7a28b5aba567487e6938cac848456dabbf05b99b7\": container with ID starting with 2299eb90b4070901ed736bd7a28b5aba567487e6938cac848456dabbf05b99b7 not found: ID does not exist" Jun 11 21:23:52.608316 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:52.608287 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-584789bccc-4gppn"] Jun 11 21:23:52.615990 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:52.615970 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-584789bccc-4gppn"] Jun 11 21:23:54.616128 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:23:54.616098 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a168313a-978a-4f8c-b3e7-4e00816ed9cc" path="/var/lib/kubelet/pods/a168313a-978a-4f8c-b3e7-4e00816ed9cc/volumes" Jun 11 21:24:11.141475 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.141436 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6bc447988d-lpcst"] Jun 11 21:24:11.142096 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.141739 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a168313a-978a-4f8c-b3e7-4e00816ed9cc" containerName="manager" Jun 11 21:24:11.142096 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.141751 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a168313a-978a-4f8c-b3e7-4e00816ed9cc" containerName="manager" Jun 11 21:24:11.142096 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.141802 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a168313a-978a-4f8c-b3e7-4e00816ed9cc" containerName="manager" Jun 11 21:24:11.146155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.146137 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6bc447988d-lpcst" Jun 11 21:24:11.148496 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.148476 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 11 21:24:11.148754 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.148737 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-n9dm8\"" Jun 11 21:24:11.158515 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.158489 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6bc447988d-lpcst"] Jun 11 21:24:11.231067 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.231006 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8126083b-ebf4-4c94-9237-3a7c860100b8-openshift-service-ca\") pod \"maas-api-6bc447988d-lpcst\" (UID: \"8126083b-ebf4-4c94-9237-3a7c860100b8\") " pod="opendatahub/maas-api-6bc447988d-lpcst" Jun 11 21:24:11.231251 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.231080 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42f74\" (UniqueName: \"kubernetes.io/projected/8126083b-ebf4-4c94-9237-3a7c860100b8-kube-api-access-42f74\") pod \"maas-api-6bc447988d-lpcst\" (UID: \"8126083b-ebf4-4c94-9237-3a7c860100b8\") " pod="opendatahub/maas-api-6bc447988d-lpcst" Jun 11 21:24:11.231251 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.231144 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8126083b-ebf4-4c94-9237-3a7c860100b8-maas-api-tls\") pod \"maas-api-6bc447988d-lpcst\" (UID: \"8126083b-ebf4-4c94-9237-3a7c860100b8\") " pod="opendatahub/maas-api-6bc447988d-lpcst" Jun 11 21:24:11.331770 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.331733 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8126083b-ebf4-4c94-9237-3a7c860100b8-maas-api-tls\") pod \"maas-api-6bc447988d-lpcst\" (UID: \"8126083b-ebf4-4c94-9237-3a7c860100b8\") " pod="opendatahub/maas-api-6bc447988d-lpcst" Jun 11 21:24:11.331951 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.331794 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8126083b-ebf4-4c94-9237-3a7c860100b8-openshift-service-ca\") pod \"maas-api-6bc447988d-lpcst\" (UID: \"8126083b-ebf4-4c94-9237-3a7c860100b8\") " pod="opendatahub/maas-api-6bc447988d-lpcst" Jun 11 21:24:11.331951 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.331825 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42f74\" (UniqueName: \"kubernetes.io/projected/8126083b-ebf4-4c94-9237-3a7c860100b8-kube-api-access-42f74\") pod \"maas-api-6bc447988d-lpcst\" (UID: \"8126083b-ebf4-4c94-9237-3a7c860100b8\") " pod="opendatahub/maas-api-6bc447988d-lpcst" Jun 11 21:24:11.332580 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.332551 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8126083b-ebf4-4c94-9237-3a7c860100b8-openshift-service-ca\") pod \"maas-api-6bc447988d-lpcst\" (UID: \"8126083b-ebf4-4c94-9237-3a7c860100b8\") " pod="opendatahub/maas-api-6bc447988d-lpcst" Jun 11 21:24:11.334216 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.334196 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8126083b-ebf4-4c94-9237-3a7c860100b8-maas-api-tls\") pod \"maas-api-6bc447988d-lpcst\" (UID: \"8126083b-ebf4-4c94-9237-3a7c860100b8\") " pod="opendatahub/maas-api-6bc447988d-lpcst" Jun 11 21:24:11.341975 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.341954 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42f74\" (UniqueName: \"kubernetes.io/projected/8126083b-ebf4-4c94-9237-3a7c860100b8-kube-api-access-42f74\") pod \"maas-api-6bc447988d-lpcst\" (UID: \"8126083b-ebf4-4c94-9237-3a7c860100b8\") " pod="opendatahub/maas-api-6bc447988d-lpcst" Jun 11 21:24:11.456209 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.456114 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6bc447988d-lpcst" Jun 11 21:24:11.595190 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.595068 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6bc447988d-lpcst"] Jun 11 21:24:11.597679 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.597650 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-f9j9s"] Jun 11 21:24:11.598277 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:24:11.598252 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8126083b_ebf4_4c94_9237_3a7c860100b8.slice/crio-99c29b09bf8d7c54ad0f5bf73e3e0fae394691563d84da53e8c0f2767e92b2af WatchSource:0}: Error finding container 99c29b09bf8d7c54ad0f5bf73e3e0fae394691563d84da53e8c0f2767e92b2af: Status 404 returned error can't find the container with id 99c29b09bf8d7c54ad0f5bf73e3e0fae394691563d84da53e8c0f2767e92b2af Jun 11 21:24:11.603703 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.603684 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-f9j9s" Jun 11 21:24:11.607177 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.607149 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 11 21:24:11.607475 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.607459 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-22rb4\"" Jun 11 21:24:11.614765 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.614738 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-f9j9s"] Jun 11 21:24:11.633477 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.633445 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdjqk\" (UniqueName: \"kubernetes.io/projected/7f339660-5225-480a-8612-a15c9d747ee2-kube-api-access-pdjqk\") pod \"payload-pre-processing-579785bc64-f9j9s\" (UID: \"7f339660-5225-480a-8612-a15c9d747ee2\") " pod="openshift-ingress/payload-pre-processing-579785bc64-f9j9s" Jun 11 21:24:11.641177 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.641142 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-n22m6"] Jun 11 21:24:11.645501 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.645480 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-n22m6" Jun 11 21:24:11.653590 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.653567 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-n22m6"] Jun 11 21:24:11.668402 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.668366 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6bc447988d-lpcst" event={"ID":"8126083b-ebf4-4c94-9237-3a7c860100b8","Type":"ContainerStarted","Data":"99c29b09bf8d7c54ad0f5bf73e3e0fae394691563d84da53e8c0f2767e92b2af"} Jun 11 21:24:11.734798 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.734705 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shfrk\" (UniqueName: \"kubernetes.io/projected/f3d7012e-d806-4945-9ef7-25be1a3f5ee3-kube-api-access-shfrk\") pod \"payload-processing-699b6cd6c-n22m6\" (UID: \"f3d7012e-d806-4945-9ef7-25be1a3f5ee3\") " pod="openshift-ingress/payload-processing-699b6cd6c-n22m6" Jun 11 21:24:11.734798 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.734758 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdjqk\" (UniqueName: \"kubernetes.io/projected/7f339660-5225-480a-8612-a15c9d747ee2-kube-api-access-pdjqk\") pod \"payload-pre-processing-579785bc64-f9j9s\" (UID: \"7f339660-5225-480a-8612-a15c9d747ee2\") " pod="openshift-ingress/payload-pre-processing-579785bc64-f9j9s" Jun 11 21:24:11.742372 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.742347 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdjqk\" (UniqueName: \"kubernetes.io/projected/7f339660-5225-480a-8612-a15c9d747ee2-kube-api-access-pdjqk\") pod \"payload-pre-processing-579785bc64-f9j9s\" (UID: \"7f339660-5225-480a-8612-a15c9d747ee2\") " pod="openshift-ingress/payload-pre-processing-579785bc64-f9j9s" Jun 11 21:24:11.835230 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.835188 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shfrk\" (UniqueName: \"kubernetes.io/projected/f3d7012e-d806-4945-9ef7-25be1a3f5ee3-kube-api-access-shfrk\") pod \"payload-processing-699b6cd6c-n22m6\" (UID: \"f3d7012e-d806-4945-9ef7-25be1a3f5ee3\") " pod="openshift-ingress/payload-processing-699b6cd6c-n22m6" Jun 11 21:24:11.842983 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.842951 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shfrk\" (UniqueName: \"kubernetes.io/projected/f3d7012e-d806-4945-9ef7-25be1a3f5ee3-kube-api-access-shfrk\") pod \"payload-processing-699b6cd6c-n22m6\" (UID: \"f3d7012e-d806-4945-9ef7-25be1a3f5ee3\") " pod="openshift-ingress/payload-processing-699b6cd6c-n22m6" Jun 11 21:24:11.915119 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.915087 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-f9j9s" Jun 11 21:24:11.956263 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:11.956225 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-n22m6" Jun 11 21:24:12.073728 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:12.073681 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-f9j9s"] Jun 11 21:24:12.077618 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:24:12.077565 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f339660_5225_480a_8612_a15c9d747ee2.slice/crio-feb633d28edaa42ef01933719d4fa90e39a5343345e30154bd198fad7da88bed WatchSource:0}: Error finding container feb633d28edaa42ef01933719d4fa90e39a5343345e30154bd198fad7da88bed: Status 404 returned error can't find the container with id feb633d28edaa42ef01933719d4fa90e39a5343345e30154bd198fad7da88bed Jun 11 21:24:12.117015 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:12.116987 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-n22m6"] Jun 11 21:24:12.120346 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:24:12.120321 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d7012e_d806_4945_9ef7_25be1a3f5ee3.slice/crio-c96b28c221904b513f50ca51180e971606a778f7806c152ca20d858eac7f16d0 WatchSource:0}: Error finding container c96b28c221904b513f50ca51180e971606a778f7806c152ca20d858eac7f16d0: Status 404 returned error can't find the container with id c96b28c221904b513f50ca51180e971606a778f7806c152ca20d858eac7f16d0 Jun 11 21:24:12.680374 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:12.680333 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-n22m6" event={"ID":"f3d7012e-d806-4945-9ef7-25be1a3f5ee3","Type":"ContainerStarted","Data":"c96b28c221904b513f50ca51180e971606a778f7806c152ca20d858eac7f16d0"} Jun 11 21:24:12.682881 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:12.682807 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-f9j9s" event={"ID":"7f339660-5225-480a-8612-a15c9d747ee2","Type":"ContainerStarted","Data":"feb633d28edaa42ef01933719d4fa90e39a5343345e30154bd198fad7da88bed"} Jun 11 21:24:13.689082 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:13.689047 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6bc447988d-lpcst" event={"ID":"8126083b-ebf4-4c94-9237-3a7c860100b8","Type":"ContainerStarted","Data":"aeaeb696d821bfcc6ae2fe08693b8aaeca472256e32f40fa8c914019f8aaaa42"} Jun 11 21:24:13.689529 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:13.689161 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6bc447988d-lpcst" Jun 11 21:24:13.707732 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:13.707675 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6bc447988d-lpcst" podStartSLOduration=0.769330085 podStartE2EDuration="2.707654401s" totalImagesPullingTime="1.938324316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:24:13.70546592 +0000 UTC m=+827.448545910" watchObservedRunningTime="2026-06-11 21:24:13.707654401 +0000 UTC m=+827.450734390" Jun 11 21:24:15.707662 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:15.707621 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-n22m6" event={"ID":"f3d7012e-d806-4945-9ef7-25be1a3f5ee3","Type":"ContainerStarted","Data":"d0a9df970d3b7b9cfcf667805073a2171111e74cdc3a451bec1fdc426422cddd"} Jun 11 21:24:15.708161 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:15.707741 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-699b6cd6c-n22m6" Jun 11 21:24:15.709259 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:15.709240 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-f9j9s" event={"ID":"7f339660-5225-480a-8612-a15c9d747ee2","Type":"ContainerStarted","Data":"ca4208425f0fa9c8ec82efc6a972d2264636436d521261979dda09e44a477fac"} Jun 11 21:24:15.709332 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:15.709289 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-579785bc64-f9j9s" Jun 11 21:24:15.724097 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:15.724054 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-699b6cd6c-n22m6" podStartSLOduration=2.212116841 podStartE2EDuration="4.72404299s" totalImagesPullingTime="2.511926149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:24:15.7234143 +0000 UTC m=+829.466494316" watchObservedRunningTime="2026-06-11 21:24:15.72404299 +0000 UTC m=+829.467122971" Jun 11 21:24:15.737437 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:15.737391 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-579785bc64-f9j9s" podStartSLOduration=2.189236573 podStartE2EDuration="4.737378618s" totalImagesPullingTime="2.548142045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:24:15.736479885 +0000 UTC m=+829.479559873" watchObservedRunningTime="2026-06-11 21:24:15.737378618 +0000 UTC m=+829.480458606" Jun 11 21:24:19.708556 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:19.708526 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6bc447988d-lpcst" Jun 11 21:24:26.715440 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:26.715409 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-699b6cd6c-n22m6" Jun 11 21:24:26.715800 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:24:26.715476 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-579785bc64-f9j9s" Jun 11 21:25:26.610059 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:25:26.609967 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:25:26.614928 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:25:26.614902 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:26:24.061139 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:24.061098 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 21:26:24.061622 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:24.061309 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" containerID="cri-o://f125e01f800682447b5cc300bac8ce07f5e513f7b290dc25bc480d3239826c3c" gracePeriod=30 Jun 11 21:26:26.108385 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.108360 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 21:26:26.198365 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.198329 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkqmk\" (UniqueName: \"kubernetes.io/projected/914d36e1-ea89-4cd1-afd9-905bbf589d92-kube-api-access-hkqmk\") pod \"914d36e1-ea89-4cd1-afd9-905bbf589d92\" (UID: \"914d36e1-ea89-4cd1-afd9-905bbf589d92\") " Jun 11 21:26:26.200531 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.200503 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/914d36e1-ea89-4cd1-afd9-905bbf589d92-kube-api-access-hkqmk" (OuterVolumeSpecName: "kube-api-access-hkqmk") pod "914d36e1-ea89-4cd1-afd9-905bbf589d92" (UID: "914d36e1-ea89-4cd1-afd9-905bbf589d92"). InnerVolumeSpecName "kube-api-access-hkqmk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:26:26.261435 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.261351 2563 generic.go:356] "Generic (PLEG): container finished" podID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerID="f125e01f800682447b5cc300bac8ce07f5e513f7b290dc25bc480d3239826c3c" exitCode=143 Jun 11 21:26:26.261435 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.261393 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 21:26:26.261640 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.261392 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"914d36e1-ea89-4cd1-afd9-905bbf589d92","Type":"ContainerDied","Data":"f125e01f800682447b5cc300bac8ce07f5e513f7b290dc25bc480d3239826c3c"} Jun 11 21:26:26.261640 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.261537 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"914d36e1-ea89-4cd1-afd9-905bbf589d92","Type":"ContainerDied","Data":"fab4b889305e3fd66986c83eb0b21f83aae25a688317ab688f327addb3e7f32a"} Jun 11 21:26:26.261640 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.261573 2563 scope.go:117] "RemoveContainer" containerID="f125e01f800682447b5cc300bac8ce07f5e513f7b290dc25bc480d3239826c3c" Jun 11 21:26:26.271300 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.271283 2563 scope.go:117] "RemoveContainer" containerID="f125e01f800682447b5cc300bac8ce07f5e513f7b290dc25bc480d3239826c3c" Jun 11 21:26:26.271565 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:26:26.271544 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f125e01f800682447b5cc300bac8ce07f5e513f7b290dc25bc480d3239826c3c\": container with ID starting with f125e01f800682447b5cc300bac8ce07f5e513f7b290dc25bc480d3239826c3c not found: ID does not exist" containerID="f125e01f800682447b5cc300bac8ce07f5e513f7b290dc25bc480d3239826c3c" Jun 11 21:26:26.271619 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.271574 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f125e01f800682447b5cc300bac8ce07f5e513f7b290dc25bc480d3239826c3c"} err="failed to get container status \"f125e01f800682447b5cc300bac8ce07f5e513f7b290dc25bc480d3239826c3c\": rpc error: code = NotFound desc = could not find container \"f125e01f800682447b5cc300bac8ce07f5e513f7b290dc25bc480d3239826c3c\": container with ID starting with f125e01f800682447b5cc300bac8ce07f5e513f7b290dc25bc480d3239826c3c not found: ID does not exist" Jun 11 21:26:26.282407 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.282376 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 21:26:26.283827 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.283808 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 21:26:26.299940 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.299920 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkqmk\" (UniqueName: \"kubernetes.io/projected/914d36e1-ea89-4cd1-afd9-905bbf589d92-kube-api-access-hkqmk\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:26:26.307336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.307315 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 21:26:26.307676 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.307662 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" Jun 11 21:26:26.307717 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.307678 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" Jun 11 21:26:26.307751 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.307746 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" containerName="keycloak" Jun 11 21:26:26.312108 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.312092 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 21:26:26.314314 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.314291 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 11 21:26:26.314522 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.314500 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 11 21:26:26.314648 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.314580 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-b4v4x\"" Jun 11 21:26:26.314648 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.314599 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 11 21:26:26.314739 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.314688 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 11 21:26:26.318593 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.318570 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 21:26:26.400512 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.400480 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf7kk\" (UniqueName: \"kubernetes.io/projected/97dee6d8-ec65-432c-8ffa-879050a4b029-kube-api-access-lf7kk\") pod \"maas-keycloak-0\" (UID: \"97dee6d8-ec65-432c-8ffa-879050a4b029\") " pod="keycloak-system/maas-keycloak-0" Jun 11 21:26:26.400718 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.400531 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/97dee6d8-ec65-432c-8ffa-879050a4b029-test-realms\") pod \"maas-keycloak-0\" (UID: \"97dee6d8-ec65-432c-8ffa-879050a4b029\") " pod="keycloak-system/maas-keycloak-0" Jun 11 21:26:26.501088 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.501063 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf7kk\" (UniqueName: \"kubernetes.io/projected/97dee6d8-ec65-432c-8ffa-879050a4b029-kube-api-access-lf7kk\") pod \"maas-keycloak-0\" (UID: \"97dee6d8-ec65-432c-8ffa-879050a4b029\") " pod="keycloak-system/maas-keycloak-0" Jun 11 21:26:26.501235 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.501103 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/97dee6d8-ec65-432c-8ffa-879050a4b029-test-realms\") pod \"maas-keycloak-0\" (UID: \"97dee6d8-ec65-432c-8ffa-879050a4b029\") " pod="keycloak-system/maas-keycloak-0" Jun 11 21:26:26.501815 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.501797 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/97dee6d8-ec65-432c-8ffa-879050a4b029-test-realms\") pod \"maas-keycloak-0\" (UID: \"97dee6d8-ec65-432c-8ffa-879050a4b029\") " pod="keycloak-system/maas-keycloak-0" Jun 11 21:26:26.509122 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.509094 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf7kk\" (UniqueName: \"kubernetes.io/projected/97dee6d8-ec65-432c-8ffa-879050a4b029-kube-api-access-lf7kk\") pod \"maas-keycloak-0\" (UID: \"97dee6d8-ec65-432c-8ffa-879050a4b029\") " pod="keycloak-system/maas-keycloak-0" Jun 11 21:26:26.617270 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.617242 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="914d36e1-ea89-4cd1-afd9-905bbf589d92" path="/var/lib/kubelet/pods/914d36e1-ea89-4cd1-afd9-905bbf589d92/volumes" Jun 11 21:26:26.624719 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.624699 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-b4v4x\"" Jun 11 21:26:26.633496 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.633482 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 21:26:26.763795 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.763769 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 21:26:26.765611 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:26:26.765583 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97dee6d8_ec65_432c_8ffa_879050a4b029.slice/crio-45a4bedbe57233cb5bd8f9631120be69a68a9dfcfb8b69029d21b2d1ace38779 WatchSource:0}: Error finding container 45a4bedbe57233cb5bd8f9631120be69a68a9dfcfb8b69029d21b2d1ace38779: Status 404 returned error can't find the container with id 45a4bedbe57233cb5bd8f9631120be69a68a9dfcfb8b69029d21b2d1ace38779 Jun 11 21:26:26.766776 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:26.766756 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 21:26:27.053009 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:27.052988 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 11 21:26:27.267585 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:27.267548 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"97dee6d8-ec65-432c-8ffa-879050a4b029","Type":"ContainerStarted","Data":"3b8bb9af4be10ce411ebc9b1c9ac9f9fdda6a1425e6f66c41886946d9f2c6805"} Jun 11 21:26:27.268067 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:27.267605 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"97dee6d8-ec65-432c-8ffa-879050a4b029","Type":"ContainerStarted","Data":"45a4bedbe57233cb5bd8f9631120be69a68a9dfcfb8b69029d21b2d1ace38779"} Jun 11 21:26:27.288078 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:27.288008 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.004603006 podStartE2EDuration="1.287992897s" totalImagesPullingTime="283.389891ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:26:27.286540559 +0000 UTC m=+961.029620548" watchObservedRunningTime="2026-06-11 21:26:27.287992897 +0000 UTC m=+961.031072916" Jun 11 21:26:27.633910 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:27.633868 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 11 21:26:27.635797 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:27.635760 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="97dee6d8-ec65-432c-8ffa-879050a4b029" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 11 21:26:28.634656 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:28.634603 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="97dee6d8-ec65-432c-8ffa-879050a4b029" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 11 21:26:29.634762 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:29.634721 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="97dee6d8-ec65-432c-8ffa-879050a4b029" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 11 21:26:30.634698 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:30.634646 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="97dee6d8-ec65-432c-8ffa-879050a4b029" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 11 21:26:31.634520 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:31.634463 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="97dee6d8-ec65-432c-8ffa-879050a4b029" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 11 21:26:32.634634 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:32.634565 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="97dee6d8-ec65-432c-8ffa-879050a4b029" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 11 21:26:33.634633 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:33.634590 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="97dee6d8-ec65-432c-8ffa-879050a4b029" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 11 21:26:34.634889 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:34.634833 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="97dee6d8-ec65-432c-8ffa-879050a4b029" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 11 21:26:35.634134 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:35.634076 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="97dee6d8-ec65-432c-8ffa-879050a4b029" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 11 21:26:36.633902 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:36.633867 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 11 21:26:36.634412 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:36.634202 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="97dee6d8-ec65-432c-8ffa-879050a4b029" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 11 21:26:37.634218 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:37.634165 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="97dee6d8-ec65-432c-8ffa-879050a4b029" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 11 21:26:38.634729 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:38.634688 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="97dee6d8-ec65-432c-8ffa-879050a4b029" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 11 21:26:39.634724 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:39.634669 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="97dee6d8-ec65-432c-8ffa-879050a4b029" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 11 21:26:40.762109 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:40.762063 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 11 21:26:40.776959 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:40.776913 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="97dee6d8-ec65-432c-8ffa-879050a4b029" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 21:26:50.769685 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:26:50.769648 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 11 21:27:32.885938 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:32.885896 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9"] Jun 11 21:27:32.889521 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:32.889502 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:32.891952 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:32.891927 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 11 21:27:32.892097 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:32.891971 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 11 21:27:32.892097 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:32.891971 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-cbzqn\"" Jun 11 21:27:32.892820 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:32.892801 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 11 21:27:32.901639 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:32.901616 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9"] Jun 11 21:27:32.911612 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:32.911592 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:32.911731 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:32.911644 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ggj4\" (UniqueName: \"kubernetes.io/projected/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-kube-api-access-6ggj4\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:32.911731 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:32.911696 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:32.911852 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:32.911786 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:32.911914 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:32.911852 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:32.911914 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:32.911884 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:33.013109 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:33.013079 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:33.013291 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:33.013142 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ggj4\" (UniqueName: \"kubernetes.io/projected/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-kube-api-access-6ggj4\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:33.013291 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:33.013173 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:33.013291 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:33.013207 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:33.013291 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:33.013262 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:33.013291 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:33.013287 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:33.013706 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:33.013682 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:33.013787 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:33.013738 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:33.013787 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:33.013738 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:33.015380 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:33.015361 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:33.015557 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:33.015538 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:33.020605 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:33.020582 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ggj4\" (UniqueName: \"kubernetes.io/projected/260b2d3a-9dc2-4a4e-8137-b3004e3bb366-kube-api-access-6ggj4\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9\" (UID: \"260b2d3a-9dc2-4a4e-8137-b3004e3bb366\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:33.201509 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:33.201414 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:27:33.359886 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:33.359845 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9"] Jun 11 21:27:33.364877 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:27:33.364842 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod260b2d3a_9dc2_4a4e_8137_b3004e3bb366.slice/crio-bb5397995d31036e04c55c8186105b80045c42f7ce35ca8d77c6e5e40defb382 WatchSource:0}: Error finding container bb5397995d31036e04c55c8186105b80045c42f7ce35ca8d77c6e5e40defb382: Status 404 returned error can't find the container with id bb5397995d31036e04c55c8186105b80045c42f7ce35ca8d77c6e5e40defb382 Jun 11 21:27:33.580978 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:33.580936 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" event={"ID":"260b2d3a-9dc2-4a4e-8137-b3004e3bb366","Type":"ContainerStarted","Data":"bb5397995d31036e04c55c8186105b80045c42f7ce35ca8d77c6e5e40defb382"} Jun 11 21:27:38.604792 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:27:38.604753 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" event={"ID":"260b2d3a-9dc2-4a4e-8137-b3004e3bb366","Type":"ContainerStarted","Data":"ce6c6bfa94b73d8d9e8d739b763124b1822b5fac11990642bad65797c10fea7c"} Jun 11 21:28:10.378128 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.378092 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs"] Jun 11 21:28:10.394766 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.394739 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs"] Jun 11 21:28:10.394905 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.394874 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.397296 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.397274 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 11 21:28:10.446405 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.446371 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/677c4753-b7ee-408a-b203-7a7a15165740-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.446556 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.446414 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/677c4753-b7ee-408a-b203-7a7a15165740-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.446556 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.446444 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/677c4753-b7ee-408a-b203-7a7a15165740-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.446556 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.446498 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6btcz\" (UniqueName: \"kubernetes.io/projected/677c4753-b7ee-408a-b203-7a7a15165740-kube-api-access-6btcz\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.446556 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.446527 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/677c4753-b7ee-408a-b203-7a7a15165740-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.446703 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.446563 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/677c4753-b7ee-408a-b203-7a7a15165740-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.547646 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.547611 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6btcz\" (UniqueName: \"kubernetes.io/projected/677c4753-b7ee-408a-b203-7a7a15165740-kube-api-access-6btcz\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.547646 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.547651 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/677c4753-b7ee-408a-b203-7a7a15165740-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.547876 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.547778 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/677c4753-b7ee-408a-b203-7a7a15165740-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.547933 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.547880 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/677c4753-b7ee-408a-b203-7a7a15165740-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.547933 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.547919 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/677c4753-b7ee-408a-b203-7a7a15165740-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.548058 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.547959 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/677c4753-b7ee-408a-b203-7a7a15165740-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.548058 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.547991 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/677c4753-b7ee-408a-b203-7a7a15165740-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.548278 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.548259 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/677c4753-b7ee-408a-b203-7a7a15165740-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.548341 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.548268 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/677c4753-b7ee-408a-b203-7a7a15165740-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.549972 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.549949 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/677c4753-b7ee-408a-b203-7a7a15165740-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.550175 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.550158 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/677c4753-b7ee-408a-b203-7a7a15165740-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.554881 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.554857 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6btcz\" (UniqueName: \"kubernetes.io/projected/677c4753-b7ee-408a-b203-7a7a15165740-kube-api-access-6btcz\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs\" (UID: \"677c4753-b7ee-408a-b203-7a7a15165740\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.705627 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.705547 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:28:10.831919 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:10.831896 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs"] Jun 11 21:28:10.834192 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:28:10.834162 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod677c4753_b7ee_408a_b203_7a7a15165740.slice/crio-8f990f528506204864ed833bfdaeebd8d7ae6a9473524120b1d10cc10b1578e2 WatchSource:0}: Error finding container 8f990f528506204864ed833bfdaeebd8d7ae6a9473524120b1d10cc10b1578e2: Status 404 returned error can't find the container with id 8f990f528506204864ed833bfdaeebd8d7ae6a9473524120b1d10cc10b1578e2 Jun 11 21:28:11.747914 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:11.747877 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" event={"ID":"677c4753-b7ee-408a-b203-7a7a15165740","Type":"ContainerStarted","Data":"265f0e3ec7dd14cd6ad6ecd404abc20ff120248189f8f330dbd2ed7604fae7f4"} Jun 11 21:28:11.748290 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:11.747937 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" event={"ID":"677c4753-b7ee-408a-b203-7a7a15165740","Type":"ContainerStarted","Data":"8f990f528506204864ed833bfdaeebd8d7ae6a9473524120b1d10cc10b1578e2"} Jun 11 21:28:29.828146 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:29.828101 2563 generic.go:356] "Generic (PLEG): container finished" podID="260b2d3a-9dc2-4a4e-8137-b3004e3bb366" containerID="ce6c6bfa94b73d8d9e8d739b763124b1822b5fac11990642bad65797c10fea7c" exitCode=0 Jun 11 21:28:29.828668 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:29.828174 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" event={"ID":"260b2d3a-9dc2-4a4e-8137-b3004e3bb366","Type":"ContainerDied","Data":"ce6c6bfa94b73d8d9e8d739b763124b1822b5fac11990642bad65797c10fea7c"} Jun 11 21:28:31.839577 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:31.839540 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" event={"ID":"260b2d3a-9dc2-4a4e-8137-b3004e3bb366","Type":"ContainerStarted","Data":"3f09397ff51b4d3c5815039e3efecb70e90ae7533ca978224d9df8caa5f08bc0"} Jun 11 21:28:31.857730 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:31.857685 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" podStartSLOduration=2.547371906 podStartE2EDuration="59.857672517s" totalImagesPullingTime="5.440804517s" totalInitContainerRuntime="51.869496094s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:27:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:28:31.857343518 +0000 UTC m=+1085.600423524" watchObservedRunningTime="2026-06-11 21:28:31.857672517 +0000 UTC m=+1085.600752503" Jun 11 21:28:33.183185 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.183148 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp"] Jun 11 21:28:33.186805 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.186783 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.189050 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.189003 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 11 21:28:33.196791 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.196769 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp"] Jun 11 21:28:33.201692 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.201669 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:28:33.201797 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.201756 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:28:33.213860 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.213842 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:28:33.250897 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.250867 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56365126-3320-40b2-8f82-4cb32b1ecb02-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.251052 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.250924 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56365126-3320-40b2-8f82-4cb32b1ecb02-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.251052 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.250965 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56365126-3320-40b2-8f82-4cb32b1ecb02-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.251148 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.251127 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56365126-3320-40b2-8f82-4cb32b1ecb02-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.251201 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.251187 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2ll6\" (UniqueName: \"kubernetes.io/projected/56365126-3320-40b2-8f82-4cb32b1ecb02-kube-api-access-g2ll6\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.251235 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.251212 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56365126-3320-40b2-8f82-4cb32b1ecb02-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.351836 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.351802 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56365126-3320-40b2-8f82-4cb32b1ecb02-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.352004 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.351849 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56365126-3320-40b2-8f82-4cb32b1ecb02-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.352004 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.351870 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56365126-3320-40b2-8f82-4cb32b1ecb02-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.352004 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.351896 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56365126-3320-40b2-8f82-4cb32b1ecb02-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.352004 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.351938 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2ll6\" (UniqueName: \"kubernetes.io/projected/56365126-3320-40b2-8f82-4cb32b1ecb02-kube-api-access-g2ll6\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.352004 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.351965 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56365126-3320-40b2-8f82-4cb32b1ecb02-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.352292 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.352267 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56365126-3320-40b2-8f82-4cb32b1ecb02-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.352350 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.352305 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56365126-3320-40b2-8f82-4cb32b1ecb02-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.352414 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.352395 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56365126-3320-40b2-8f82-4cb32b1ecb02-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.354426 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.354402 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56365126-3320-40b2-8f82-4cb32b1ecb02-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.354530 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.354456 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56365126-3320-40b2-8f82-4cb32b1ecb02-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.358899 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.358877 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2ll6\" (UniqueName: \"kubernetes.io/projected/56365126-3320-40b2-8f82-4cb32b1ecb02-kube-api-access-g2ll6\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp\" (UID: \"56365126-3320-40b2-8f82-4cb32b1ecb02\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.497268 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.497178 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:28:33.622135 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.622112 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp"] Jun 11 21:28:33.624141 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:28:33.624111 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56365126_3320_40b2_8f82_4cb32b1ecb02.slice/crio-720d5a2dbea6f6f4928c5e5a4e911d5b01658d15ba961203a7521543336c2f57 WatchSource:0}: Error finding container 720d5a2dbea6f6f4928c5e5a4e911d5b01658d15ba961203a7521543336c2f57: Status 404 returned error can't find the container with id 720d5a2dbea6f6f4928c5e5a4e911d5b01658d15ba961203a7521543336c2f57 Jun 11 21:28:33.850429 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.850396 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" event={"ID":"56365126-3320-40b2-8f82-4cb32b1ecb02","Type":"ContainerStarted","Data":"b269c364c3c7c92e0ed59219c0df4a010eb2c4bb0f6eae6f128f4572fa161dea"} Jun 11 21:28:33.850622 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.850455 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" event={"ID":"56365126-3320-40b2-8f82-4cb32b1ecb02","Type":"ContainerStarted","Data":"720d5a2dbea6f6f4928c5e5a4e911d5b01658d15ba961203a7521543336c2f57"} Jun 11 21:28:33.864625 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:28:33.864597 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9" Jun 11 21:29:10.008398 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:10.008313 2563 generic.go:356] "Generic (PLEG): container finished" podID="677c4753-b7ee-408a-b203-7a7a15165740" containerID="265f0e3ec7dd14cd6ad6ecd404abc20ff120248189f8f330dbd2ed7604fae7f4" exitCode=0 Jun 11 21:29:10.008398 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:10.008354 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" event={"ID":"677c4753-b7ee-408a-b203-7a7a15165740","Type":"ContainerDied","Data":"265f0e3ec7dd14cd6ad6ecd404abc20ff120248189f8f330dbd2ed7604fae7f4"} Jun 11 21:29:11.014758 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:11.014724 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" event={"ID":"677c4753-b7ee-408a-b203-7a7a15165740","Type":"ContainerStarted","Data":"7733aab5aa1c888ae20e1a124bd80f54b206fc8cd19731c8ac38e81f9be3944c"} Jun 11 21:29:11.033943 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:11.033894 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" podStartSLOduration=1.746452568 podStartE2EDuration="1m1.033880702s" totalImagesPullingTime="420.717876ms" totalInitContainerRuntime="58.866710258s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:28:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:29:11.0328073 +0000 UTC m=+1124.775887299" watchObservedRunningTime="2026-06-11 21:29:11.033880702 +0000 UTC m=+1124.776960689" Jun 11 21:29:20.706647 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:20.706614 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:29:20.707237 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:20.706677 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:29:20.718758 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:20.718735 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:29:21.065768 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:21.065745 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs" Jun 11 21:29:28.086305 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:28.086266 2563 generic.go:356] "Generic (PLEG): container finished" podID="56365126-3320-40b2-8f82-4cb32b1ecb02" containerID="b269c364c3c7c92e0ed59219c0df4a010eb2c4bb0f6eae6f128f4572fa161dea" exitCode=0 Jun 11 21:29:28.086680 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:28.086320 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" event={"ID":"56365126-3320-40b2-8f82-4cb32b1ecb02","Type":"ContainerDied","Data":"b269c364c3c7c92e0ed59219c0df4a010eb2c4bb0f6eae6f128f4572fa161dea"} Jun 11 21:29:29.092651 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:29.092618 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" event={"ID":"56365126-3320-40b2-8f82-4cb32b1ecb02","Type":"ContainerStarted","Data":"afee3d8093752511533dd08f679feea5f097ee0d34341d28223f785abd086b62"} Jun 11 21:29:29.112056 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:29.111991 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" podStartSLOduration=2.21286327 podStartE2EDuration="56.111974566s" totalImagesPullingTime="178.223242ms" totalInitContainerRuntime="53.720888054s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:28:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:29:29.110128749 +0000 UTC m=+1142.853208735" watchObservedRunningTime="2026-06-11 21:29:29.111974566 +0000 UTC m=+1142.855054552" Jun 11 21:29:33.497691 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:33.497648 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:29:33.498102 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:33.497721 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:29:33.510248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:33.510215 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:29:34.125254 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:29:34.125227 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp" Jun 11 21:30:26.638419 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:30:26.638392 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:30:26.643937 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:30:26.643912 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:31:10.444628 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:10.444591 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74864b9f6-kvtjf"] Jun 11 21:31:10.445162 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:10.444832 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-74864b9f6-kvtjf" podUID="91dac085-89c5-40d5-a981-a4fd0b03f449" containerName="manager" containerID="cri-o://5011cd184b1a3b7e90d418f8dbc3c504ed028e8c1353bfacea16f7109f091d21" gracePeriod=30 Jun 11 21:31:10.692716 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:10.692689 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74864b9f6-kvtjf" Jun 11 21:31:10.817947 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:10.817922 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/91dac085-89c5-40d5-a981-a4fd0b03f449-webhook-cert\") pod \"91dac085-89c5-40d5-a981-a4fd0b03f449\" (UID: \"91dac085-89c5-40d5-a981-a4fd0b03f449\") " Jun 11 21:31:10.818123 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:10.817977 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4rlj\" (UniqueName: \"kubernetes.io/projected/91dac085-89c5-40d5-a981-a4fd0b03f449-kube-api-access-t4rlj\") pod \"91dac085-89c5-40d5-a981-a4fd0b03f449\" (UID: \"91dac085-89c5-40d5-a981-a4fd0b03f449\") " Jun 11 21:31:10.819906 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:10.819878 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91dac085-89c5-40d5-a981-a4fd0b03f449-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "91dac085-89c5-40d5-a981-a4fd0b03f449" (UID: "91dac085-89c5-40d5-a981-a4fd0b03f449"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:31:10.820005 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:10.819988 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91dac085-89c5-40d5-a981-a4fd0b03f449-kube-api-access-t4rlj" (OuterVolumeSpecName: "kube-api-access-t4rlj") pod "91dac085-89c5-40d5-a981-a4fd0b03f449" (UID: "91dac085-89c5-40d5-a981-a4fd0b03f449"). InnerVolumeSpecName "kube-api-access-t4rlj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:31:10.918971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:10.918934 2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/91dac085-89c5-40d5-a981-a4fd0b03f449-webhook-cert\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:31:10.918971 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:10.918965 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4rlj\" (UniqueName: \"kubernetes.io/projected/91dac085-89c5-40d5-a981-a4fd0b03f449-kube-api-access-t4rlj\") on node \"ip-10-0-140-167.ec2.internal\" DevicePath \"\"" Jun 11 21:31:11.534323 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:11.534287 2563 generic.go:356] "Generic (PLEG): container finished" podID="91dac085-89c5-40d5-a981-a4fd0b03f449" containerID="5011cd184b1a3b7e90d418f8dbc3c504ed028e8c1353bfacea16f7109f091d21" exitCode=0 Jun 11 21:31:11.534712 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:11.534332 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74864b9f6-kvtjf" Jun 11 21:31:11.534712 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:11.534367 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74864b9f6-kvtjf" event={"ID":"91dac085-89c5-40d5-a981-a4fd0b03f449","Type":"ContainerDied","Data":"5011cd184b1a3b7e90d418f8dbc3c504ed028e8c1353bfacea16f7109f091d21"} Jun 11 21:31:11.534712 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:11.534420 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74864b9f6-kvtjf" event={"ID":"91dac085-89c5-40d5-a981-a4fd0b03f449","Type":"ContainerDied","Data":"13117089d2636494adafd3d040794f00f09c0d8fa624f45e58e4e49edad84110"} Jun 11 21:31:11.534712 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:11.534441 2563 scope.go:117] "RemoveContainer" containerID="5011cd184b1a3b7e90d418f8dbc3c504ed028e8c1353bfacea16f7109f091d21" Jun 11 21:31:11.543200 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:11.543184 2563 scope.go:117] "RemoveContainer" containerID="5011cd184b1a3b7e90d418f8dbc3c504ed028e8c1353bfacea16f7109f091d21" Jun 11 21:31:11.543441 ip-10-0-140-167 kubenswrapper[2563]: E0611 21:31:11.543424 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5011cd184b1a3b7e90d418f8dbc3c504ed028e8c1353bfacea16f7109f091d21\": container with ID starting with 5011cd184b1a3b7e90d418f8dbc3c504ed028e8c1353bfacea16f7109f091d21 not found: ID does not exist" containerID="5011cd184b1a3b7e90d418f8dbc3c504ed028e8c1353bfacea16f7109f091d21" Jun 11 21:31:11.543500 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:11.543449 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5011cd184b1a3b7e90d418f8dbc3c504ed028e8c1353bfacea16f7109f091d21"} err="failed to get container status \"5011cd184b1a3b7e90d418f8dbc3c504ed028e8c1353bfacea16f7109f091d21\": rpc error: code = NotFound desc = could not find container \"5011cd184b1a3b7e90d418f8dbc3c504ed028e8c1353bfacea16f7109f091d21\": container with ID starting with 5011cd184b1a3b7e90d418f8dbc3c504ed028e8c1353bfacea16f7109f091d21 not found: ID does not exist" Jun 11 21:31:11.554337 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:11.554315 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74864b9f6-kvtjf"] Jun 11 21:31:11.558211 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:11.558192 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74864b9f6-kvtjf"] Jun 11 21:31:12.312905 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.312873 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74864b9f6-9f8rv"] Jun 11 21:31:12.313225 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.313213 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91dac085-89c5-40d5-a981-a4fd0b03f449" containerName="manager" Jun 11 21:31:12.313277 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.313227 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="91dac085-89c5-40d5-a981-a4fd0b03f449" containerName="manager" Jun 11 21:31:12.313314 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.313283 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="91dac085-89c5-40d5-a981-a4fd0b03f449" containerName="manager" Jun 11 21:31:12.317543 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.317528 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74864b9f6-9f8rv" Jun 11 21:31:12.319787 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.319766 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 11 21:31:12.319899 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.319769 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 11 21:31:12.320543 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.320527 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-slhdv\"" Jun 11 21:31:12.324064 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.324044 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74864b9f6-9f8rv"] Jun 11 21:31:12.431401 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.431372 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dab53b75-2081-43a6-afd8-292db8f3eefb-webhook-cert\") pod \"maas-controller-74864b9f6-9f8rv\" (UID: \"dab53b75-2081-43a6-afd8-292db8f3eefb\") " pod="opendatahub/maas-controller-74864b9f6-9f8rv" Jun 11 21:31:12.431534 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.431415 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t89vp\" (UniqueName: \"kubernetes.io/projected/dab53b75-2081-43a6-afd8-292db8f3eefb-kube-api-access-t89vp\") pod \"maas-controller-74864b9f6-9f8rv\" (UID: \"dab53b75-2081-43a6-afd8-292db8f3eefb\") " pod="opendatahub/maas-controller-74864b9f6-9f8rv" Jun 11 21:31:12.532199 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.532176 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dab53b75-2081-43a6-afd8-292db8f3eefb-webhook-cert\") pod \"maas-controller-74864b9f6-9f8rv\" (UID: \"dab53b75-2081-43a6-afd8-292db8f3eefb\") " pod="opendatahub/maas-controller-74864b9f6-9f8rv" Jun 11 21:31:12.532344 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.532217 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t89vp\" (UniqueName: \"kubernetes.io/projected/dab53b75-2081-43a6-afd8-292db8f3eefb-kube-api-access-t89vp\") pod \"maas-controller-74864b9f6-9f8rv\" (UID: \"dab53b75-2081-43a6-afd8-292db8f3eefb\") " pod="opendatahub/maas-controller-74864b9f6-9f8rv" Jun 11 21:31:12.534337 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.534312 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dab53b75-2081-43a6-afd8-292db8f3eefb-webhook-cert\") pod \"maas-controller-74864b9f6-9f8rv\" (UID: \"dab53b75-2081-43a6-afd8-292db8f3eefb\") " pod="opendatahub/maas-controller-74864b9f6-9f8rv" Jun 11 21:31:12.540618 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.540597 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t89vp\" (UniqueName: \"kubernetes.io/projected/dab53b75-2081-43a6-afd8-292db8f3eefb-kube-api-access-t89vp\") pod \"maas-controller-74864b9f6-9f8rv\" (UID: \"dab53b75-2081-43a6-afd8-292db8f3eefb\") " pod="opendatahub/maas-controller-74864b9f6-9f8rv" Jun 11 21:31:12.617224 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.617171 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91dac085-89c5-40d5-a981-a4fd0b03f449" path="/var/lib/kubelet/pods/91dac085-89c5-40d5-a981-a4fd0b03f449/volumes" Jun 11 21:31:12.628472 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.628454 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74864b9f6-9f8rv" Jun 11 21:31:12.762207 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:12.762180 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74864b9f6-9f8rv"] Jun 11 21:31:12.764077 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:31:12.764048 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab53b75_2081_43a6_afd8_292db8f3eefb.slice/crio-19dd11bc2241f366ead5d8d7cbb398f4458d594d1a6c4c1f7c88feaff559bdaa WatchSource:0}: Error finding container 19dd11bc2241f366ead5d8d7cbb398f4458d594d1a6c4c1f7c88feaff559bdaa: Status 404 returned error can't find the container with id 19dd11bc2241f366ead5d8d7cbb398f4458d594d1a6c4c1f7c88feaff559bdaa Jun 11 21:31:13.545684 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:13.545603 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74864b9f6-9f8rv" event={"ID":"dab53b75-2081-43a6-afd8-292db8f3eefb","Type":"ContainerStarted","Data":"05d2e604d9912f690b03b267e5c10d8dbad00e3ad281ab21fc9ba51dc400d0c5"} Jun 11 21:31:13.545684 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:13.545663 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74864b9f6-9f8rv" event={"ID":"dab53b75-2081-43a6-afd8-292db8f3eefb","Type":"ContainerStarted","Data":"19dd11bc2241f366ead5d8d7cbb398f4458d594d1a6c4c1f7c88feaff559bdaa"} Jun 11 21:31:13.546079 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:13.545695 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74864b9f6-9f8rv" Jun 11 21:31:13.573252 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:13.573203 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74864b9f6-9f8rv" podStartSLOduration=1.136716576 podStartE2EDuration="1.573188247s" totalImagesPullingTime="436.471671ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:31:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:31:13.571043861 +0000 UTC m=+1247.314123856" watchObservedRunningTime="2026-06-11 21:31:13.573188247 +0000 UTC m=+1247.316268233" Jun 11 21:31:24.555550 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:31:24.555466 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74864b9f6-9f8rv" Jun 11 21:35:26.664097 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:35:26.664066 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:35:26.670987 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:35:26.670960 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:40:26.690493 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:40:26.690466 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:40:26.697323 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:40:26.697301 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:45:26.716732 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:45:26.716702 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:45:26.723772 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:45:26.723751 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:50:26.742675 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:50:26.742573 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:50:26.750487 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:50:26.750460 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:51:35.785780 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:35.785746 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6bc447988d-lpcst_8126083b-ebf4-4c94-9237-3a7c860100b8/maas-api/0.log" Jun 11 21:51:36.118353 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:36.118320 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74864b9f6-9f8rv_dab53b75-2081-43a6-afd8-292db8f3eefb/manager/0.log" Jun 11 21:51:36.599860 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:36.599833 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66c5c8d746-jpksn_a4b01aa7-5f6b-4f71-870d-d9043ce38c61/copy-manifests/0.log" Jun 11 21:51:36.606601 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:36.606579 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66c5c8d746-jpksn_a4b01aa7-5f6b-4f71-870d-d9043ce38c61/manager/0.log" Jun 11 21:51:36.840192 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:36.840162 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-jrp4h_f3b07c49-faaf-49ea-bd22-23102814a7b2/postgres/0.log" Jun 11 21:51:38.464994 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:38.464962 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-r4zn2_525d6a7a-f7d7-445d-a489-9b9f888fce17/kuadrant-console-plugin/0.log" Jun 11 21:51:39.653692 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:39.653665 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c9dfcd5c9-64xrk_abd6e8a3-e983-47bb-a937-890763c5eb26/kube-auth-proxy/0.log" Jun 11 21:51:39.903232 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:39.903196 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-f9j9s_7f339660-5225-480a-8612-a15c9d747ee2/payload-pre-processing/0.log" Jun 11 21:51:40.030719 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:40.030645 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-n22m6_f3d7012e-d806-4945-9ef7-25be1a3f5ee3/payload-processing/0.log" Jun 11 21:51:40.152750 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:40.152724 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d798db97b-wnl55_e4baf798-247a-4237-a103-aa1dae1b1557/router/0.log" Jun 11 21:51:40.499779 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:40.499749 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9_260b2d3a-9dc2-4a4e-8137-b3004e3bb366/storage-initializer/0.log" Jun 11 21:51:40.507883 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:40.507860 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-k8kl9_260b2d3a-9dc2-4a4e-8137-b3004e3bb366/main/0.log" Jun 11 21:51:40.863302 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:40.863271 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs_677c4753-b7ee-408a-b203-7a7a15165740/storage-initializer/0.log" Jun 11 21:51:40.870141 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:40.870111 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477ctmqbs_677c4753-b7ee-408a-b203-7a7a15165740/main/0.log" Jun 11 21:51:40.985696 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:40.985664 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp_56365126-3320-40b2-8f82-4cb32b1ecb02/storage-initializer/0.log" Jun 11 21:51:40.992407 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:40.992385 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-cmdfp_56365126-3320-40b2-8f82-4cb32b1ecb02/main/0.log" Jun 11 21:51:48.445912 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:48.445873 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bqqv5_58f726ff-e37a-497f-8a7c-9216791bdd00/global-pull-secret-syncer/0.log" Jun 11 21:51:48.567746 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:48.567702 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l4pjg_aea1a453-0093-47c5-87bb-67c9fa59021c/konnectivity-agent/0.log" Jun 11 21:51:48.659695 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:48.659666 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-167.ec2.internal_5b0fb3102504571ccdfcf8b539950afa/haproxy/0.log" Jun 11 21:51:53.708203 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:53.708172 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-r4zn2_525d6a7a-f7d7-445d-a489-9b9f888fce17/kuadrant-console-plugin/0.log" Jun 11 21:51:55.766775 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:55.766748 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-566f7cc5bb-p4wjz_d42f6c3d-971b-4ebd-b47a-5fed2c8e07f2/metrics-server/0.log" Jun 11 21:51:55.846550 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:55.846488 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-67vf2_e41a1472-8802-45ed-a95d-88814c60cdd1/node-exporter/0.log" Jun 11 21:51:55.860655 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:55.860628 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-67vf2_e41a1472-8802-45ed-a95d-88814c60cdd1/kube-rbac-proxy/0.log" Jun 11 21:51:55.881880 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:55.881856 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-67vf2_e41a1472-8802-45ed-a95d-88814c60cdd1/init-textfile/0.log" Jun 11 21:51:56.199240 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:56.199211 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_20261d7f-29da-4d6f-9d71-2b0b30b31c43/prometheus/0.log" Jun 11 21:51:56.218808 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:56.218778 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_20261d7f-29da-4d6f-9d71-2b0b30b31c43/config-reloader/0.log" Jun 11 21:51:56.239673 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:56.239649 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_20261d7f-29da-4d6f-9d71-2b0b30b31c43/thanos-sidecar/0.log" Jun 11 21:51:56.261708 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:56.261690 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_20261d7f-29da-4d6f-9d71-2b0b30b31c43/kube-rbac-proxy-web/0.log" Jun 11 21:51:56.289898 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:56.289878 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_20261d7f-29da-4d6f-9d71-2b0b30b31c43/kube-rbac-proxy/0.log" Jun 11 21:51:56.312951 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:56.312911 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_20261d7f-29da-4d6f-9d71-2b0b30b31c43/kube-rbac-proxy-thanos/0.log" Jun 11 21:51:56.335403 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:56.335342 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_20261d7f-29da-4d6f-9d71-2b0b30b31c43/init-config-reloader/0.log" Jun 11 21:51:58.657482 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:58.657447 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/2.log" Jun 11 21:51:58.664688 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:51:58.664637 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-tnsxc_2075f225-0880-4abf-807b-378d2d3be472/console-operator/3.log" Jun 11 21:52:00.009797 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.009763 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg"] Jun 11 21:52:00.012248 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.012223 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.014609 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.014588 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jg56s\"/\"kube-root-ca.crt\"" Jun 11 21:52:00.015507 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.015487 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jg56s\"/\"default-dockercfg-ms2kw\"" Jun 11 21:52:00.015591 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.015537 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jg56s\"/\"openshift-service-ca.crt\"" Jun 11 21:52:00.024140 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.024114 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg"] Jun 11 21:52:00.120593 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.120562 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvkdj\" (UniqueName: \"kubernetes.io/projected/1cf2d6d5-749d-464a-b79b-db3fbc5bb666-kube-api-access-tvkdj\") pod \"perf-node-gather-daemonset-vb8zg\" (UID: \"1cf2d6d5-749d-464a-b79b-db3fbc5bb666\") " pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.120760 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.120616 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1cf2d6d5-749d-464a-b79b-db3fbc5bb666-podres\") pod \"perf-node-gather-daemonset-vb8zg\" (UID: \"1cf2d6d5-749d-464a-b79b-db3fbc5bb666\") " pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.120760 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.120689 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1cf2d6d5-749d-464a-b79b-db3fbc5bb666-lib-modules\") pod \"perf-node-gather-daemonset-vb8zg\" (UID: \"1cf2d6d5-749d-464a-b79b-db3fbc5bb666\") " pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.120760 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.120751 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1cf2d6d5-749d-464a-b79b-db3fbc5bb666-proc\") pod \"perf-node-gather-daemonset-vb8zg\" (UID: \"1cf2d6d5-749d-464a-b79b-db3fbc5bb666\") " pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.120868 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.120769 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1cf2d6d5-749d-464a-b79b-db3fbc5bb666-sys\") pod \"perf-node-gather-daemonset-vb8zg\" (UID: \"1cf2d6d5-749d-464a-b79b-db3fbc5bb666\") " pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.221506 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.221468 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1cf2d6d5-749d-464a-b79b-db3fbc5bb666-podres\") pod \"perf-node-gather-daemonset-vb8zg\" (UID: \"1cf2d6d5-749d-464a-b79b-db3fbc5bb666\") " pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.221506 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.221514 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1cf2d6d5-749d-464a-b79b-db3fbc5bb666-lib-modules\") pod \"perf-node-gather-daemonset-vb8zg\" (UID: \"1cf2d6d5-749d-464a-b79b-db3fbc5bb666\") " pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.221718 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.221567 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1cf2d6d5-749d-464a-b79b-db3fbc5bb666-proc\") pod \"perf-node-gather-daemonset-vb8zg\" (UID: \"1cf2d6d5-749d-464a-b79b-db3fbc5bb666\") " pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.221718 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.221585 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1cf2d6d5-749d-464a-b79b-db3fbc5bb666-sys\") pod \"perf-node-gather-daemonset-vb8zg\" (UID: \"1cf2d6d5-749d-464a-b79b-db3fbc5bb666\") " pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.221718 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.221619 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvkdj\" (UniqueName: \"kubernetes.io/projected/1cf2d6d5-749d-464a-b79b-db3fbc5bb666-kube-api-access-tvkdj\") pod \"perf-node-gather-daemonset-vb8zg\" (UID: \"1cf2d6d5-749d-464a-b79b-db3fbc5bb666\") " pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.221718 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.221648 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1cf2d6d5-749d-464a-b79b-db3fbc5bb666-podres\") pod \"perf-node-gather-daemonset-vb8zg\" (UID: \"1cf2d6d5-749d-464a-b79b-db3fbc5bb666\") " pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.221718 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.221672 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1cf2d6d5-749d-464a-b79b-db3fbc5bb666-proc\") pod \"perf-node-gather-daemonset-vb8zg\" (UID: \"1cf2d6d5-749d-464a-b79b-db3fbc5bb666\") " pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.221718 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.221697 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1cf2d6d5-749d-464a-b79b-db3fbc5bb666-sys\") pod \"perf-node-gather-daemonset-vb8zg\" (UID: \"1cf2d6d5-749d-464a-b79b-db3fbc5bb666\") " pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.221961 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.221720 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1cf2d6d5-749d-464a-b79b-db3fbc5bb666-lib-modules\") pod \"perf-node-gather-daemonset-vb8zg\" (UID: \"1cf2d6d5-749d-464a-b79b-db3fbc5bb666\") " pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.229167 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.229137 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvkdj\" (UniqueName: \"kubernetes.io/projected/1cf2d6d5-749d-464a-b79b-db3fbc5bb666-kube-api-access-tvkdj\") pod \"perf-node-gather-daemonset-vb8zg\" (UID: \"1cf2d6d5-749d-464a-b79b-db3fbc5bb666\") " pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.321894 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.321863 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:00.451200 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.451172 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg"] Jun 11 21:52:00.453739 ip-10-0-140-167 kubenswrapper[2563]: W0611 21:52:00.453715 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1cf2d6d5_749d_464a_b79b_db3fbc5bb666.slice/crio-0447f1193f979a337a74883d9fe88016aedd942126cd7c1915c13b401e10cd1b WatchSource:0}: Error finding container 0447f1193f979a337a74883d9fe88016aedd942126cd7c1915c13b401e10cd1b: Status 404 returned error can't find the container with id 0447f1193f979a337a74883d9fe88016aedd942126cd7c1915c13b401e10cd1b Jun 11 21:52:00.997712 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.997616 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" event={"ID":"1cf2d6d5-749d-464a-b79b-db3fbc5bb666","Type":"ContainerStarted","Data":"8af1aa874a21ed1f59cda2b3daec9e6fd21d8e4cf894e2f0b39115c09bd12aec"} Jun 11 21:52:00.997712 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.997673 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" event={"ID":"1cf2d6d5-749d-464a-b79b-db3fbc5bb666","Type":"ContainerStarted","Data":"0447f1193f979a337a74883d9fe88016aedd942126cd7c1915c13b401e10cd1b"} Jun 11 21:52:00.997943 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:00.997805 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:01.016093 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:01.015687 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" podStartSLOduration=2.015663551 podStartE2EDuration="2.015663551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:51:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:52:01.013235807 +0000 UTC m=+2494.756315797" watchObservedRunningTime="2026-06-11 21:52:01.015663551 +0000 UTC m=+2494.758743541" Jun 11 21:52:01.220318 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:01.220287 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5qxs8_faf3cd10-7238-4409-bf88-932cd0b1c14e/dns/0.log" Jun 11 21:52:01.239923 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:01.239902 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5qxs8_faf3cd10-7238-4409-bf88-932cd0b1c14e/kube-rbac-proxy/0.log" Jun 11 21:52:01.305823 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:01.305802 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9wcfm_a76fe3c1-133a-4c0d-b36f-4f309a013579/dns-node-resolver/0.log" Jun 11 21:52:01.869390 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:01.869353 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-84589dccd5-zrb5r_15b77cd7-0a18-40d8-8c07-2dd97a915b5e/registry/0.log" Jun 11 21:52:01.884428 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:01.884399 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-84589dccd5-zrb5r_15b77cd7-0a18-40d8-8c07-2dd97a915b5e/registry/1.log" Jun 11 21:52:01.957057 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:01.957030 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vb7pc_23d74d65-50c4-45b2-bfb0-33bd224279c8/node-ca/0.log" Jun 11 21:52:03.029876 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:03.029848 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c9dfcd5c9-64xrk_abd6e8a3-e983-47bb-a937-890763c5eb26/kube-auth-proxy/0.log" Jun 11 21:52:03.149351 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:03.149316 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-f9j9s_7f339660-5225-480a-8612-a15c9d747ee2/payload-pre-processing/0.log" Jun 11 21:52:03.249917 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:03.249884 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-n22m6_f3d7012e-d806-4945-9ef7-25be1a3f5ee3/payload-processing/0.log" Jun 11 21:52:03.271504 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:03.271472 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d798db97b-wnl55_e4baf798-247a-4237-a103-aa1dae1b1557/router/0.log" Jun 11 21:52:03.979945 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:03.979922 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zvmqc_99fddcb2-b296-415e-94a9-15a40dc7462d/serve-healthcheck-canary/0.log" Jun 11 21:52:04.613994 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:04.613959 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x9k48_3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5/kube-rbac-proxy/0.log" Jun 11 21:52:04.632530 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:04.632505 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x9k48_3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5/exporter/0.log" Jun 11 21:52:04.657565 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:04.657535 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x9k48_3f13317f-8f3c-41c5-bd66-3b0cccb0a1e5/extractor/0.log" Jun 11 21:52:07.011969 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:07.011940 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jg56s/perf-node-gather-daemonset-vb8zg" Jun 11 21:52:07.057336 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:07.057309 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6bc447988d-lpcst_8126083b-ebf4-4c94-9237-3a7c860100b8/maas-api/0.log" Jun 11 21:52:07.213357 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:07.213327 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74864b9f6-9f8rv_dab53b75-2081-43a6-afd8-292db8f3eefb/manager/0.log" Jun 11 21:52:07.507682 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:07.507650 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66c5c8d746-jpksn_a4b01aa7-5f6b-4f71-870d-d9043ce38c61/manager/0.log" Jun 11 21:52:07.531385 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:07.531355 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66c5c8d746-jpksn_a4b01aa7-5f6b-4f71-870d-d9043ce38c61/copy-manifests/0.log" Jun 11 21:52:07.598181 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:07.598149 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-jrp4h_f3b07c49-faaf-49ea-bd22-23102814a7b2/postgres/0.log" Jun 11 21:52:09.238829 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:09.238753 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5dffd8664c-k5drx_9674668a-7839-4a01-9bdb-10292c7225c2/manager/0.log" Jun 11 21:52:13.508168 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:13.508136 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c9dfcd5c9-64xrk_abd6e8a3-e983-47bb-a937-890763c5eb26/kube-auth-proxy/0.log" Jun 11 21:52:13.638078 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:13.638051 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-f9j9s_7f339660-5225-480a-8612-a15c9d747ee2/payload-pre-processing/0.log" Jun 11 21:52:13.738590 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:13.738559 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-n22m6_f3d7012e-d806-4945-9ef7-25be1a3f5ee3/payload-processing/0.log" Jun 11 21:52:13.756756 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:13.756728 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d798db97b-wnl55_e4baf798-247a-4237-a103-aa1dae1b1557/router/0.log" Jun 11 21:52:15.009944 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:15.009912 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c9dfcd5c9-64xrk_abd6e8a3-e983-47bb-a937-890763c5eb26/kube-auth-proxy/0.log" Jun 11 21:52:15.111193 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:15.111161 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-f9j9s_7f339660-5225-480a-8612-a15c9d747ee2/payload-pre-processing/0.log" Jun 11 21:52:15.193945 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:15.193906 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-n22m6_f3d7012e-d806-4945-9ef7-25be1a3f5ee3/payload-processing/0.log" Jun 11 21:52:15.209450 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:15.209423 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d798db97b-wnl55_e4baf798-247a-4237-a103-aa1dae1b1557/router/0.log" Jun 11 21:52:16.404129 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:16.404052 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mkwd5_38607a8f-f08d-4fce-a21a-c885cc677d51/kube-multus-additional-cni-plugins/0.log" Jun 11 21:52:16.421155 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:16.421127 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mkwd5_38607a8f-f08d-4fce-a21a-c885cc677d51/egress-router-binary-copy/0.log" Jun 11 21:52:16.438848 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:16.438826 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mkwd5_38607a8f-f08d-4fce-a21a-c885cc677d51/cni-plugins/0.log" Jun 11 21:52:16.456330 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:16.456308 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mkwd5_38607a8f-f08d-4fce-a21a-c885cc677d51/bond-cni-plugin/0.log" Jun 11 21:52:16.477357 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:16.477334 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mkwd5_38607a8f-f08d-4fce-a21a-c885cc677d51/routeoverride-cni/0.log" Jun 11 21:52:16.494686 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:16.494664 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mkwd5_38607a8f-f08d-4fce-a21a-c885cc677d51/whereabouts-cni-bincopy/0.log" Jun 11 21:52:16.512229 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:16.512203 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mkwd5_38607a8f-f08d-4fce-a21a-c885cc677d51/whereabouts-cni/0.log" Jun 11 21:52:16.714282 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:16.714207 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vtjdn_7e44a0b9-6458-4700-8d71-d45f13e8ad3b/kube-multus/0.log" Jun 11 21:52:16.807002 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:16.806977 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hz8r8_db4e8904-7461-4943-909f-6e9fe29dea61/network-metrics-daemon/0.log" Jun 11 21:52:16.824199 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:16.824174 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c9dfcd5c9-64xrk_abd6e8a3-e983-47bb-a937-890763c5eb26/kube-auth-proxy/0.log" Jun 11 21:52:16.826617 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:16.826600 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hz8r8_db4e8904-7461-4943-909f-6e9fe29dea61/kube-rbac-proxy/0.log" Jun 11 21:52:16.936643 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:16.936601 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-f9j9s_7f339660-5225-480a-8612-a15c9d747ee2/payload-pre-processing/0.log" Jun 11 21:52:17.018871 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:17.018805 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-n22m6_f3d7012e-d806-4945-9ef7-25be1a3f5ee3/payload-processing/0.log" Jun 11 21:52:17.055497 ip-10-0-140-167 kubenswrapper[2563]: I0611 21:52:17.055465 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d798db97b-wnl55_e4baf798-247a-4237-a103-aa1dae1b1557/router/0.log"