Jun 15 03:06:10.909122 ip-10-0-139-167 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 15 03:06:10.909133 ip-10-0-139-167 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 15 03:06:10.909140 ip-10-0-139-167 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 15 03:06:10.909369 ip-10-0-139-167 systemd[1]: Failed to start Kubernetes Kubelet. Jun 15 03:06:20.918290 ip-10-0-139-167 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 15 03:06:20.918304 ip-10-0-139-167 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 02573ec54f334ad7b0cd8556f2cddcd8 -- Jun 15 03:08:41.731319 ip-10-0-139-167 systemd[1]: Starting Kubernetes Kubelet... Jun 15 03:08:42.095805 ip-10-0-139-167 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 15 03:08:42.095805 ip-10-0-139-167 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 15 03:08:42.095805 ip-10-0-139-167 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 15 03:08:42.095805 ip-10-0-139-167 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 15 03:08:42.095805 ip-10-0-139-167 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 15 03:08:42.097102 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.095863 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 15 03:08:42.102301 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102285 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 03:08:42.102301 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102301 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102304 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102309 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102312 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102316 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102326 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102330 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102333 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102337 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102341 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102344 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102347 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102350 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102354 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102356 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102359 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102362 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102364 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102367 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 03:08:42.102372 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102369 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102385 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102388 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102391 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102393 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102396 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102399 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102401 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102404 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102406 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102409 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102412 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102414 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102417 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102420 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102423 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102425 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102428 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102430 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 03:08:42.102853 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102434 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102437 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102440 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102442 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102445 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102447 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102450 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102452 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102455 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102457 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102460 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102462 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102465 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102467 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102470 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102474 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102477 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102479 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102482 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102485 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 03:08:42.103320 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102487 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102490 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102493 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102495 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102498 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102500 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102504 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102506 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102509 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102512 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102514 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102518 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102521 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102524 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102527 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102530 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102533 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102535 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102538 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 03:08:42.103832 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102541 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102543 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102546 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102548 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102551 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102553 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102555 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102558 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102561 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102565 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102567 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102570 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102572 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102575 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102577 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102580 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102582 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102585 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102587 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102590 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 03:08:42.104311 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.102592 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103303 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103310 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103313 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103316 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103319 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103323 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103326 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103329 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103332 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103335 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103338 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103340 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103344 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103348 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103350 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103354 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103356 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103359 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103361 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 03:08:42.104831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103366 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103369 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103372 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103374 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103377 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103380 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103382 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103385 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103387 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103390 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103392 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103395 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103398 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103400 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103403 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103405 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103408 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103410 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103413 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103415 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 03:08:42.105410 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103418 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103420 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103423 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103426 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103430 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103434 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103436 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103440 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103442 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103445 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103448 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103450 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103454 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103457 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103460 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103463 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103466 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103468 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103471 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 03:08:42.105893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103473 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103476 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103479 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103481 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103484 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103486 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103489 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103492 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103494 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103496 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103499 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103502 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103504 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103507 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103510 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103512 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103516 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103518 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103521 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 03:08:42.106389 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103523 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103526 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103528 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103531 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103533 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103536 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103539 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103541 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103544 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103546 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103549 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103552 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103554 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103557 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103559 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103562 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103564 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103568 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103571 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103574 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 03:08:42.106852 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103577 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103580 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103934 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103938 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103941 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103944 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103947 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103950 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103952 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103955 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103958 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103960 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103963 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103965 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103968 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103971 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103973 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103976 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103979 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103982 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 03:08:42.107361 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103984 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103987 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103990 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103993 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.103997 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104000 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104003 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104007 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104010 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104013 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104016 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104018 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104021 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104023 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104026 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104029 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104032 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104035 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104037 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 03:08:42.107833 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104040 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104043 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104045 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104047 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104050 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104053 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104056 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104059 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104061 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104064 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104067 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104069 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104072 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104074 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104077 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104095 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104100 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104105 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104108 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104113 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 03:08:42.108317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104116 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104118 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104121 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104123 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104126 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104128 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104132 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104135 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104137 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104141 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104144 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104147 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104150 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104152 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104155 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104158 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104161 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104163 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104166 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 03:08:42.108813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104168 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104171 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104174 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104176 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104179 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104181 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104183 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104186 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104189 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104192 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104195 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104197 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104200 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104205 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104208 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104211 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104213 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104216 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104219 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104221 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 03:08:42.109300 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104228 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104231 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104233 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104303 2560 flags.go:64] FLAG: --address="0.0.0.0" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104309 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104316 2560 flags.go:64] FLAG: --anonymous-auth="true" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104321 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104325 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104329 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104333 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104337 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104340 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104343 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104346 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104350 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104353 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104356 2560 flags.go:64] FLAG: --cgroup-root="" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104359 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104362 2560 flags.go:64] FLAG: --client-ca-file="" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104366 2560 flags.go:64] FLAG: --cloud-provider="external" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104369 2560 flags.go:64] FLAG: --cluster-dns="[]" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104373 2560 flags.go:64] FLAG: --cluster-domain="" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104375 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104379 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 15 03:08:42.109787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104382 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104385 2560 flags.go:64] FLAG: --container-log-max-files="5" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104389 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104392 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104395 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104398 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104401 2560 flags.go:64] FLAG: --contention-profiling="false" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104404 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104408 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104411 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104414 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104417 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104420 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104423 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104426 2560 flags.go:64] FLAG: --enable-load-reader="false" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104429 2560 flags.go:64] FLAG: --enable-server="true" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104432 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104437 2560 flags.go:64] FLAG: --event-burst="100" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104440 2560 flags.go:64] FLAG: --event-qps="50" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104443 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104445 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104448 2560 flags.go:64] FLAG: --eviction-hard="" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104452 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104455 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104458 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 15 03:08:42.110377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104461 2560 flags.go:64] FLAG: --eviction-soft="" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104464 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104467 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104469 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104472 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104475 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104478 2560 flags.go:64] FLAG: --fail-swap-on="true" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104484 2560 flags.go:64] FLAG: --feature-gates="" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104488 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104491 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104494 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104497 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104500 2560 flags.go:64] FLAG: --healthz-port="10248" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104502 2560 flags.go:64] FLAG: --help="false" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104505 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104508 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104512 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104515 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104518 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104521 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104524 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104527 2560 flags.go:64] FLAG: --image-service-endpoint="" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104530 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 15 03:08:42.111000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104533 2560 flags.go:64] FLAG: --kube-api-burst="100" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104536 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104539 2560 flags.go:64] FLAG: --kube-api-qps="50" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104542 2560 flags.go:64] FLAG: --kube-reserved="" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104545 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104548 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104551 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104554 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104557 2560 flags.go:64] FLAG: --lock-file="" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104560 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104564 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104567 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104572 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104575 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104577 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104580 2560 flags.go:64] FLAG: --logging-format="text" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104584 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104588 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104590 2560 flags.go:64] FLAG: --manifest-url="" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104593 2560 flags.go:64] FLAG: --manifest-url-header="" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104598 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104601 2560 flags.go:64] FLAG: --max-open-files="1000000" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104604 2560 flags.go:64] FLAG: --max-pods="110" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104607 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104610 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 15 03:08:42.111751 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104614 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104617 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104620 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104623 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104626 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104633 2560 flags.go:64] FLAG: --node-status-max-images="50" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104636 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104640 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104644 2560 flags.go:64] FLAG: --pod-cidr="" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104646 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4a2e90c8ef7d783e4d635e24fb09435f5e1f6e954583512c0944f9e57bc77b52" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104651 2560 flags.go:64] FLAG: --pod-manifest-path="" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104654 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104657 2560 flags.go:64] FLAG: --pods-per-core="0" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104661 2560 flags.go:64] FLAG: --port="10250" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104664 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104667 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e3376663f1b371c9" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104670 2560 flags.go:64] FLAG: --qos-reserved="" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104673 2560 flags.go:64] FLAG: --read-only-port="10255" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104676 2560 flags.go:64] FLAG: --register-node="true" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104679 2560 flags.go:64] FLAG: --register-with-taints="" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104682 2560 flags.go:64] FLAG: --registry-burst="10" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104685 2560 flags.go:64] FLAG: --registry-qps="5" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104688 2560 flags.go:64] FLAG: --reserved-cpus="" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104691 2560 flags.go:64] FLAG: --reserved-memory="" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104695 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 15 03:08:42.112622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104699 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104702 2560 flags.go:64] FLAG: --rotate-certificates="false" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104704 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104707 2560 flags.go:64] FLAG: --runonce="false" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104710 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104713 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104727 2560 flags.go:64] FLAG: --seccomp-default="false" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104730 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104733 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104736 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104739 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104742 2560 flags.go:64] FLAG: --storage-driver-password="root" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104745 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104748 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104751 2560 flags.go:64] FLAG: --storage-driver-user="root" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104753 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104757 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104759 2560 flags.go:64] FLAG: --system-cgroups="" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104762 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104767 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104770 2560 flags.go:64] FLAG: --tls-cert-file="" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104773 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104777 2560 flags.go:64] FLAG: --tls-min-version="" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104779 2560 flags.go:64] FLAG: --tls-private-key-file="" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104782 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jun 15 03:08:42.113278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104785 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104787 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104791 2560 flags.go:64] FLAG: --v="2" Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104799 2560 flags.go:64] FLAG: --version="false" Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104803 2560 flags.go:64] FLAG: --vmodule="" Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104807 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.104810 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104919 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104924 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104927 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104929 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104931 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104934 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104937 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104944 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104947 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104950 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104954 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104958 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104961 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104966 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 03:08:42.113877 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104969 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104972 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104974 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104977 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104980 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104983 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104985 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104988 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104991 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104993 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104996 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.104998 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105001 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105003 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105006 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105008 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105011 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105013 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105016 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105018 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 03:08:42.114426 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105021 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105024 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105027 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105029 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105032 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105036 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105038 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105041 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105043 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105046 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105048 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105052 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105055 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105058 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105060 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105063 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105065 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105068 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105072 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 03:08:42.114941 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105075 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105092 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105095 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105098 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105101 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105103 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105106 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105109 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105112 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105114 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105117 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105119 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105122 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105124 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105127 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105130 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105132 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105135 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105139 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105142 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 03:08:42.115438 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105144 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105147 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105149 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105152 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105159 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105161 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105164 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105166 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105169 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105172 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105174 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105177 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105180 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105183 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105185 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105188 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105190 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105193 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105195 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105198 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 03:08:42.115930 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105201 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 03:08:42.116434 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105203 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 03:08:42.116434 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105206 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 03:08:42.116434 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105208 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 03:08:42.116434 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105210 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 03:08:42.116434 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.105213 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 03:08:42.116434 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.105857 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 15 03:08:42.116434 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.113550 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 15 03:08:42.116434 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.113662 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 15 03:08:42.116434 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.113684 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 15 03:08:42.116434 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.113690 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 15 03:08:42.116434 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113747 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 03:08:42.116434 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113752 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 03:08:42.116434 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113756 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 03:08:42.116434 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113760 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113763 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113766 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113768 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113771 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113774 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113777 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113780 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113782 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113785 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113788 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113791 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113793 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113796 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113799 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113801 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113804 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113807 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113809 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113812 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 03:08:42.116802 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113815 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113818 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113820 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113823 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113827 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113829 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113832 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113835 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113838 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113840 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113843 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113845 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113848 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113850 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113853 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113855 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113858 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113862 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113867 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 03:08:42.117339 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113869 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113873 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113876 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113879 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113882 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113885 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113888 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113891 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113894 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113897 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113900 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113902 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113904 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113907 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113909 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113912 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113914 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113917 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113919 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113923 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 03:08:42.117831 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113926 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113928 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113931 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113935 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113938 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113941 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113944 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113946 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113949 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113951 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113954 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113957 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113959 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113961 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113964 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113966 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113969 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113972 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113974 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113977 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 03:08:42.118428 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113979 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113982 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113984 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113987 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113989 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113992 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113994 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113997 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.113999 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114002 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114006 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114010 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114014 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114017 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114020 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114023 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 03:08:42.119021 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114025 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 03:08:42.119474 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.114031 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 15 03:08:42.119474 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114160 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 03:08:42.119474 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114166 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 03:08:42.119474 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114169 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 03:08:42.119474 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114172 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 03:08:42.119474 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114175 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 03:08:42.119474 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114178 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 03:08:42.119474 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114181 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 03:08:42.119474 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114184 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 03:08:42.119474 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114187 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 03:08:42.119474 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114190 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 03:08:42.119474 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114193 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 03:08:42.119474 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114195 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 03:08:42.119474 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114199 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 03:08:42.119474 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114201 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114204 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114206 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114209 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114211 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114214 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114217 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114219 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114221 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114224 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114226 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114229 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114232 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114235 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114238 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114240 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114243 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114245 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114248 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114251 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 03:08:42.119859 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114253 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114256 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114259 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114261 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114263 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114266 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114269 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114271 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114274 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114276 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114279 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114281 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114285 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114287 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114290 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114292 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114295 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114297 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114300 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114303 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 03:08:42.120447 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114305 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114308 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114310 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114314 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114326 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114330 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114333 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114336 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114339 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114342 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114345 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114347 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114350 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114353 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114355 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114357 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114361 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114364 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114367 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 03:08:42.121246 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114370 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114372 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114376 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114379 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114381 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114384 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114387 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114390 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114393 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114395 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114398 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114401 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114403 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114406 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114408 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114411 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114414 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114416 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114419 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114421 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 03:08:42.121863 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114424 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 03:08:42.122634 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114426 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 03:08:42.122634 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114429 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 03:08:42.122634 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114432 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 03:08:42.122634 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114435 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 03:08:42.122634 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114438 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 03:08:42.122634 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:42.114440 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 03:08:42.122634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.114445 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 15 03:08:42.122634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.114550 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jun 15 03:08:42.122634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.116785 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 15 03:08:42.122634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.118337 2560 server.go:1019] "Starting client certificate rotation" Jun 15 03:08:42.122634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.118453 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 15 03:08:42.122634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.118502 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 15 03:08:42.122634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.121514 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 15 03:08:42.126728 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.126713 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 15 03:08:42.154544 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.154522 2560 log.go:25] "Validated CRI v1 runtime API" Jun 15 03:08:42.158304 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.158289 2560 log.go:25] "Validated CRI v1 image API" Jun 15 03:08:42.159058 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.159045 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 15 03:08:42.161966 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.161943 2560 fs.go:135] Filesystem UUIDs: map[1b9ad488-1d43-40d7-860b-a5d1e62ad59e:/dev/nvme0n1p4 5aab2c53-a355-415f-825c-9272929d60ff:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jun 15 03:08:42.162019 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.161967 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 15 03:08:42.163219 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.163200 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 15 03:08:42.168116 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.167986 2560 manager.go:216] Machine: {Timestamp:2026-06-15 03:08:42.166682825 +0000 UTC m=+0.323297024 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099254 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f957cb37c2b246bebb2f9cbba47c2 SystemUUID:ec2f957c-b37c-2b24-6beb-b2f9cbba47c2 BootID:02573ec5-4f33-4ad7-b0cd-8556f2cddcd8 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4f:fe:e5:12:b3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4f:fe:e5:12:b3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:9a:45:b2:ee:19 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 15 03:08:42.168116 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.168108 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 15 03:08:42.168282 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.168266 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.120.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260608-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 15 03:08:42.171051 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.171020 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 15 03:08:42.171189 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.171053 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-167.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 15 03:08:42.172507 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.172497 2560 topology_manager.go:138] "Creating topology manager with none policy" Jun 15 03:08:42.172540 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.172510 2560 container_manager_linux.go:309] "Creating device plugin manager" Jun 15 03:08:42.172540 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.172518 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 15 03:08:42.173148 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.173138 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 15 03:08:42.174611 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.174601 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 15 03:08:42.177841 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.177830 2560 state_mem.go:36] "Initialized new in-memory state store" Jun 15 03:08:42.177944 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.177935 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 15 03:08:42.177974 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.177963 2560 kubelet.go:486] "Attempting to sync node with API server" Jun 15 03:08:42.177974 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.177972 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 15 03:08:42.178024 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.177982 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 15 03:08:42.178024 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.177991 2560 kubelet.go:398] "Adding apiserver pod source" Jun 15 03:08:42.178024 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.178000 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 15 03:08:42.178360 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.178342 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 15 03:08:42.178649 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.178627 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 15 03:08:42.180656 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.180637 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 15 03:08:42.182241 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.182225 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 15 03:08:42.182337 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.182264 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 15 03:08:42.182466 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.182452 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 15 03:08:42.182527 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.182471 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 15 03:08:42.182527 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.182481 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 15 03:08:42.182527 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.182490 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 15 03:08:42.182527 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.182501 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 15 03:08:42.182527 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.182509 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 15 03:08:42.183261 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.183250 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 15 03:08:42.183321 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.183262 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 15 03:08:42.183321 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.183272 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 15 03:08:42.183321 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.183281 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 15 03:08:42.183321 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.183295 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 15 03:08:42.183321 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.183307 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 15 03:08:42.184071 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.184061 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 15 03:08:42.184142 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.184074 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 15 03:08:42.187068 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.187053 2560 server.go:1268] "Started kubelet" Jun 15 03:08:42.187243 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.187179 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 15 03:08:42.187336 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.187204 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 15 03:08:42.187397 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.187342 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 15 03:08:42.187953 ip-10-0-139-167 systemd[1]: Started Kubernetes Kubelet. Jun 15 03:08:42.189112 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.189074 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-167.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 15 03:08:42.189249 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.189229 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-167.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 15 03:08:42.189308 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.189299 2560 server.go:310] "Adding debug handlers to kubelet server" Jun 15 03:08:42.189357 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.189313 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 15 03:08:42.189414 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.189395 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 15 03:08:42.193442 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.192600 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-167.ec2.internal.18b9225fbcd49676 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-167.ec2.internal,UID:ip-10-0-139-167.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-167.ec2.internal,},FirstTimestamp:2026-06-15 03:08:42.187028086 +0000 UTC m=+0.343642284,LastTimestamp:2026-06-15 03:08:42.187028086 +0000 UTC m=+0.343642284,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-167.ec2.internal,}" Jun 15 03:08:42.195201 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.195184 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 15 03:08:42.195882 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.195868 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 15 03:08:42.195935 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.195891 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 15 03:08:42.195997 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.195930 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jun 15 03:08:42.196515 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.196495 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 15 03:08:42.196515 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.196497 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 15 03:08:42.196651 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.196523 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 15 03:08:42.196651 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.196594 2560 factory.go:55] Registering systemd factory Jun 15 03:08:42.196651 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.196609 2560 factory.go:223] Registration of the systemd container factory successfully Jun 15 03:08:42.196651 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.196610 2560 reconstruct.go:97] "Volume reconstruction finished" Jun 15 03:08:42.196651 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.196621 2560 reconciler.go:29] "Reconciler: start to sync state" Jun 15 03:08:42.197309 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.197216 2560 factory.go:153] Registering CRI-O factory Jun 15 03:08:42.197309 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.197233 2560 factory.go:223] Registration of the crio container factory successfully Jun 15 03:08:42.197309 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.197284 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 15 03:08:42.197309 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.197306 2560 factory.go:103] Registering Raw factory Jun 15 03:08:42.197731 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.197322 2560 manager.go:1196] Started watching for new ooms in manager Jun 15 03:08:42.197731 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.197685 2560 manager.go:349] Starting recovery of all containers Jun 15 03:08:42.199361 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.199334 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 15 03:08:42.199448 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.199360 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-167.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 15 03:08:42.207707 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.207685 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pcpdq" Jun 15 03:08:42.209679 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.209665 2560 manager.go:354] Recovery completed Jun 15 03:08:42.213674 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.213660 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pcpdq" Jun 15 03:08:42.213823 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.213812 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 03:08:42.215676 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.215661 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientMemory" Jun 15 03:08:42.215744 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.215688 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 03:08:42.215744 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.215702 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientPID" Jun 15 03:08:42.216180 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.216166 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 15 03:08:42.216180 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.216178 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 15 03:08:42.216298 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.216195 2560 state_mem.go:36] "Initialized new in-memory state store" Jun 15 03:08:42.216298 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.216195 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 03:08:42.216992 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.216974 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientMemory" Jun 15 03:08:42.217134 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.217005 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 03:08:42.217134 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.217019 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientPID" Jun 15 03:08:42.217244 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.217181 2560 policy_none.go:49] "None policy: Start" Jun 15 03:08:42.217244 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.217217 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 15 03:08:42.217244 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.217228 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 15 03:08:42.218025 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.217968 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-167.ec2.internal.18b9225fbe89b624 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-167.ec2.internal,UID:ip-10-0-139-167.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-167.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-167.ec2.internal,},FirstTimestamp:2026-06-15 03:08:42.215675428 +0000 UTC m=+0.372289626,LastTimestamp:2026-06-15 03:08:42.215675428 +0000 UTC m=+0.372289626,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-167.ec2.internal,}" Jun 15 03:08:42.218194 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.218181 2560 policy_none.go:47] "Start" Jun 15 03:08:42.252668 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.252652 2560 manager.go:337] "Starting Device Plugin manager" Jun 15 03:08:42.265745 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.252708 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 15 03:08:42.265745 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.252721 2560 server.go:87] "Starting device plugin registration server" Jun 15 03:08:42.265745 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.253029 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 15 03:08:42.265745 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.253041 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 15 03:08:42.265745 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.253148 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 15 03:08:42.265745 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.253223 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 15 03:08:42.265745 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.253229 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 15 03:08:42.265745 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.254100 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 15 03:08:42.265745 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.254204 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-167.ec2.internal\" not found" Jun 15 03:08:42.265745 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.259871 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 15 03:08:42.265745 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.261007 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 15 03:08:42.265745 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.261042 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 15 03:08:42.265745 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.261059 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jun 15 03:08:42.265745 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.261133 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 15 03:08:42.265745 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.262862 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 03:08:42.353681 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.353608 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 03:08:42.354560 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.354537 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientMemory" Jun 15 03:08:42.354646 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.354578 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 03:08:42.354646 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.354617 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientPID" Jun 15 03:08:42.354725 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.354652 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.362157 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.362136 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal"] Jun 15 03:08:42.362231 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.362211 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 03:08:42.363197 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.363174 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientMemory" Jun 15 03:08:42.363304 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.363203 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 03:08:42.363304 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.363213 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientPID" Jun 15 03:08:42.363377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.363339 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.363377 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.363357 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-167.ec2.internal\": node \"ip-10-0-139-167.ec2.internal\" not found" Jun 15 03:08:42.364423 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.364411 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 03:08:42.364615 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.364602 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.364680 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.364637 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 03:08:42.365148 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.365129 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientMemory" Jun 15 03:08:42.365217 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.365155 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientMemory" Jun 15 03:08:42.365217 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.365178 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 03:08:42.365217 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.365193 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientPID" Jun 15 03:08:42.365306 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.365161 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 03:08:42.365306 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.365250 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientPID" Jun 15 03:08:42.366446 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.366433 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.366559 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.366455 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 03:08:42.367139 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.367123 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientMemory" Jun 15 03:08:42.367215 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.367152 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 03:08:42.367215 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.367161 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientPID" Jun 15 03:08:42.391456 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.391437 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jun 15 03:08:42.397664 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.397647 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/68d6905ca08151f8a81b89181a1d6d46-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal\" (UID: \"68d6905ca08151f8a81b89181a1d6d46\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.397733 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.397671 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68d6905ca08151f8a81b89181a1d6d46-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal\" (UID: \"68d6905ca08151f8a81b89181a1d6d46\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.397733 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.397687 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3ccc0e83fda6561eb368da988cc50ea9-config\") pod \"kube-apiserver-proxy-ip-10-0-139-167.ec2.internal\" (UID: \"3ccc0e83fda6561eb368da988cc50ea9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.399781 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.399759 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-167.ec2.internal\" not found" node="ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.404576 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.404558 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-167.ec2.internal\" not found" node="ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.491737 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.491715 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jun 15 03:08:42.497977 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.497960 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/68d6905ca08151f8a81b89181a1d6d46-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal\" (UID: \"68d6905ca08151f8a81b89181a1d6d46\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.498044 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.497983 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68d6905ca08151f8a81b89181a1d6d46-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal\" (UID: \"68d6905ca08151f8a81b89181a1d6d46\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.498044 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.498000 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3ccc0e83fda6561eb368da988cc50ea9-config\") pod \"kube-apiserver-proxy-ip-10-0-139-167.ec2.internal\" (UID: \"3ccc0e83fda6561eb368da988cc50ea9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.498044 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.498025 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3ccc0e83fda6561eb368da988cc50ea9-config\") pod \"kube-apiserver-proxy-ip-10-0-139-167.ec2.internal\" (UID: \"3ccc0e83fda6561eb368da988cc50ea9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.498177 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.498055 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/68d6905ca08151f8a81b89181a1d6d46-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal\" (UID: \"68d6905ca08151f8a81b89181a1d6d46\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.498177 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.498063 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68d6905ca08151f8a81b89181a1d6d46-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal\" (UID: \"68d6905ca08151f8a81b89181a1d6d46\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.592128 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.592100 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jun 15 03:08:42.692947 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.692863 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jun 15 03:08:42.703282 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.703263 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.709379 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:42.709354 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" Jun 15 03:08:42.793304 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.793270 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jun 15 03:08:42.893843 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.893816 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jun 15 03:08:42.994394 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:42.994306 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jun 15 03:08:43.095042 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:43.095008 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jun 15 03:08:43.118502 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:43.118475 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 15 03:08:43.119125 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:43.118614 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 15 03:08:43.196178 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:43.196150 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jun 15 03:08:43.218829 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:43.218796 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-14 03:03:42 +0000 UTC" deadline="2027-12-12 08:45:25.40055973 +0000 UTC" Jun 15 03:08:43.218829 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:43.218824 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13085h36m42.181738087s" Jun 15 03:08:43.222347 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:43.222320 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 03:08:43.272759 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:43.272732 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68d6905ca08151f8a81b89181a1d6d46.slice/crio-348b92b24434f944e50a05b63a984c94864ce6be96fb8415c89c896be13a872a WatchSource:0}: Error finding container 348b92b24434f944e50a05b63a984c94864ce6be96fb8415c89c896be13a872a: Status 404 returned error can't find the container with id 348b92b24434f944e50a05b63a984c94864ce6be96fb8415c89c896be13a872a Jun 15 03:08:43.273158 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:43.273135 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ccc0e83fda6561eb368da988cc50ea9.slice/crio-1c0264ee627004f83a0362e998d2046f608c6f7471331654478930c74dadbe78 WatchSource:0}: Error finding container 1c0264ee627004f83a0362e998d2046f608c6f7471331654478930c74dadbe78: Status 404 returned error can't find the container with id 1c0264ee627004f83a0362e998d2046f608c6f7471331654478930c74dadbe78 Jun 15 03:08:43.279957 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:43.279941 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 03:08:43.296373 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:43.296354 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jun 15 03:08:43.396688 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:43.396665 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jun 15 03:08:43.436586 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:43.436568 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 03:08:43.496942 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:43.496916 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jun 15 03:08:43.510360 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:43.510343 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 15 03:08:43.511892 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:43.511880 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" Jun 15 03:08:43.520194 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:43.520152 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 15 03:08:43.577314 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:43.577295 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 03:08:44.099560 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.099533 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 03:08:44.179782 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.179755 2560 apiserver.go:52] "Watching apiserver" Jun 15 03:08:44.188282 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.188256 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 15 03:08:44.188839 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.188647 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal","openshift-network-diagnostics/network-check-target-zm8j4","kube-system/global-pull-secret-syncer-66qb8","openshift-cluster-node-tuning-operator/tuned-d5x4c","openshift-multus/multus-additional-cni-plugins-rzn4l","openshift-multus/multus-bz9qb","openshift-multus/network-metrics-daemon-9z4f4","openshift-network-operator/iptables-alerter-qfn58","openshift-ovn-kubernetes/ovnkube-node-vsz59","kube-system/konnectivity-agent-gc7b7","kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt","openshift-dns/node-resolver-zq7r8","openshift-image-registry/node-ca-vzjh4"] Jun 15 03:08:44.190109 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.190069 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:44.190214 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:44.190164 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:08:44.191116 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.191074 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zq7r8" Jun 15 03:08:44.192414 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.192392 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:44.192520 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:44.192466 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:08:44.193679 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.193657 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6zjqj\"" Jun 15 03:08:44.193771 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.193740 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 15 03:08:44.193830 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.193740 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 15 03:08:44.194269 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.193918 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.195187 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.195168 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.196211 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.196191 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 15 03:08:44.197058 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.196396 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 15 03:08:44.197058 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.196490 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zrjq6\"" Jun 15 03:08:44.197058 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.196588 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 15 03:08:44.197058 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.196834 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.197511 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.197493 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-69lfj\"" Jun 15 03:08:44.197591 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.197500 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 15 03:08:44.197644 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.197625 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 15 03:08:44.198095 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.197977 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 15 03:08:44.198095 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.197987 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 15 03:08:44.198239 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.198204 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 15 03:08:44.198979 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.198824 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.200104 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.199728 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 15 03:08:44.200930 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.200648 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hl9q5\"" Jun 15 03:08:44.200930 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.200750 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vzjh4" Jun 15 03:08:44.201988 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.201931 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-48zxw\"" Jun 15 03:08:44.203250 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.202983 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 15 03:08:44.203250 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.203019 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 15 03:08:44.203250 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.203038 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 15 03:08:44.203250 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.202984 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 15 03:08:44.203497 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.203360 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 15 03:08:44.204044 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.204018 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:08:44.204152 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:44.204118 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:08:44.204225 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.204161 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 15 03:08:44.204277 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.204226 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 15 03:08:44.204409 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.204391 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 15 03:08:44.204745 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.204724 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fq5tt\"" Jun 15 03:08:44.205369 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.205039 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 15 03:08:44.205937 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.205919 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gc7b7" Jun 15 03:08:44.206598 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.206496 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-multus-cni-dir\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.206598 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.206532 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-multus-conf-dir\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.206598 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.206555 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-systemd-units\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.206785 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.206617 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-cni-netd\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.206785 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.206652 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/04ea9897-2b15-4a68-93cf-cad10b6c7245-dbus\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:44.206785 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.206677 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-modprobe-d\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.206785 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.206700 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-sysconfig\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.206785 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.206756 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-run-systemd\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.206785 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.206779 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8f448e18-ba99-4279-b13a-b86003e4e2df-ovnkube-config\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.207068 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.206863 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8f448e18-ba99-4279-b13a-b86003e4e2df-ovnkube-script-lib\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.207068 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.206892 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rc4w\" (UniqueName: \"kubernetes.io/projected/ff14c0e9-ddf7-4720-ac87-54a88354948e-kube-api-access-5rc4w\") pod \"network-metrics-daemon-9z4f4\" (UID: \"ff14c0e9-ddf7-4720-ac87-54a88354948e\") " pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:44.207068 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.206933 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd7848c5-9bec-441f-bda7-a249bd472e1b-cnibin\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.207068 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.206960 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpdqr\" (UniqueName: \"kubernetes.io/projected/48bd8c21-f7a6-47fb-adda-1f4d03b37c64-kube-api-access-xpdqr\") pod \"node-resolver-zq7r8\" (UID: \"48bd8c21-f7a6-47fb-adda-1f4d03b37c64\") " pod="openshift-dns/node-resolver-zq7r8" Jun 15 03:08:44.207068 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.206985 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs\") pod \"network-metrics-daemon-9z4f4\" (UID: \"ff14c0e9-ddf7-4720-ac87-54a88354948e\") " pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:44.207068 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207026 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-var-lib-kubelet\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.207068 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207048 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-hostroot\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.207368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207093 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-host\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.207368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207115 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-node-log\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.207368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207137 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-etc-openvswitch\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.207368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207157 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-run\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.207368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207179 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-lib-modules\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.207368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207214 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd7848c5-9bec-441f-bda7-a249bd472e1b-os-release\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.207368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207234 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8f448e18-ba99-4279-b13a-b86003e4e2df-env-overrides\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.207368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207253 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:44.207368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207272 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd7848c5-9bec-441f-bda7-a249bd472e1b-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.207368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207311 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-cni-binary-copy\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.207368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207347 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-var-lib-cni-bin\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.207368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207370 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48bd8c21-f7a6-47fb-adda-1f4d03b37c64-tmp-dir\") pod \"node-resolver-zq7r8\" (UID: \"48bd8c21-f7a6-47fb-adda-1f4d03b37c64\") " pod="openshift-dns/node-resolver-zq7r8" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207389 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-slash\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207431 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-run-openvswitch\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207455 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-run-netns\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207486 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-var-lib-openvswitch\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207512 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/04ea9897-2b15-4a68-93cf-cad10b6c7245-kubelet-config\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207536 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-sysctl-conf\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207558 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-sysctl-d\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207610 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgbjf\" (UniqueName: \"kubernetes.io/projected/eec37bd8-a2d8-4d88-ade6-428188535543-kube-api-access-mgbjf\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207644 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-multus-socket-dir-parent\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207666 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-var-lib-kubelet\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207681 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-log-socket\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207698 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd7848c5-9bec-441f-bda7-a249bd472e1b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207737 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-cnibin\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207753 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-var-lib-cni-multus\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207767 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-cni-bin\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.207872 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207784 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8f448e18-ba99-4279-b13a-b86003e4e2df-ovn-node-metrics-cert\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207810 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eec37bd8-a2d8-4d88-ade6-428188535543-tmp\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207834 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd7848c5-9bec-441f-bda7-a249bd472e1b-system-cni-dir\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207853 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-os-release\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207881 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65pqv\" (UniqueName: \"kubernetes.io/projected/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-kube-api-access-65pqv\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207906 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/48bd8c21-f7a6-47fb-adda-1f4d03b37c64-hosts-file\") pod \"node-resolver-zq7r8\" (UID: \"48bd8c21-f7a6-47fb-adda-1f4d03b37c64\") " pod="openshift-dns/node-resolver-zq7r8" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207921 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-kubelet\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207959 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9htqq\" (UniqueName: \"kubernetes.io/projected/8f448e18-ba99-4279-b13a-b86003e4e2df-kube-api-access-9htqq\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.207991 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-run-multus-certs\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208013 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-etc-kubernetes\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208034 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-run-ovn\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208057 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bd7848c5-9bec-441f-bda7-a249bd472e1b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208095 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-sys\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208135 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-systemd\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208156 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-multus-daemon-config\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208181 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bd7848c5-9bec-441f-bda7-a249bd472e1b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.208672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208198 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-run-k8s-cni-cncf-io\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.209366 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208220 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-run-ovn-kubernetes\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.209366 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208243 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.209366 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208263 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-kubernetes\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.209366 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208278 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eec37bd8-a2d8-4d88-ade6-428188535543-etc-tuned\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.209366 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208297 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-run-netns\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.209366 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208320 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdczb\" (UniqueName: \"kubernetes.io/projected/bd7848c5-9bec-441f-bda7-a249bd472e1b-kube-api-access-tdczb\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.209366 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208349 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-system-cni-dir\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.209366 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208485 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 15 03:08:44.209366 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208674 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 15 03:08:44.209366 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208706 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 15 03:08:44.209366 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.208805 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-m48wh\"" Jun 15 03:08:44.209906 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.209576 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qfn58" Jun 15 03:08:44.211148 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.210658 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.211956 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.211934 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 15 03:08:44.212143 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.212120 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 15 03:08:44.213210 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.213003 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 15 03:08:44.213210 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.213019 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hdx9v\"" Jun 15 03:08:44.213210 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.213178 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 15 03:08:44.213460 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.213280 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nb6z4\"" Jun 15 03:08:44.213494 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.213476 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 15 03:08:44.213533 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.213506 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 15 03:08:44.228006 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.227988 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-b66nz" Jun 15 03:08:44.236361 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.236343 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-b66nz" Jun 15 03:08:44.266992 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.266945 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" event={"ID":"3ccc0e83fda6561eb368da988cc50ea9","Type":"ContainerStarted","Data":"1c0264ee627004f83a0362e998d2046f608c6f7471331654478930c74dadbe78"} Jun 15 03:08:44.268961 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.268882 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" event={"ID":"68d6905ca08151f8a81b89181a1d6d46","Type":"ContainerStarted","Data":"348b92b24434f944e50a05b63a984c94864ce6be96fb8415c89c896be13a872a"} Jun 15 03:08:44.298052 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.297871 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 15 03:08:44.308600 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308572 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-var-lib-kubelet\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.308700 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308607 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-hostroot\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.308700 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308633 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-host\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.308700 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308657 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-node-log\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.308700 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308674 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-var-lib-kubelet\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.308902 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308752 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-host\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.308902 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308720 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-node-log\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.308902 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308736 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-hostroot\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.308902 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308682 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-etc-openvswitch\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.308902 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308801 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-etc-openvswitch\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.308902 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308825 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-run\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.308902 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308851 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-lib-modules\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.308902 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308876 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd7848c5-9bec-441f-bda7-a249bd472e1b-os-release\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.308902 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308901 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8f448e18-ba99-4279-b13a-b86003e4e2df-env-overrides\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308901 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-run\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308925 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308948 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd7848c5-9bec-441f-bda7-a249bd472e1b-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308962 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd7848c5-9bec-441f-bda7-a249bd472e1b-os-release\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308970 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-cni-binary-copy\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.308993 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-lib-modules\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309012 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-var-lib-cni-bin\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309035 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48bd8c21-f7a6-47fb-adda-1f4d03b37c64-tmp-dir\") pod \"node-resolver-zq7r8\" (UID: \"48bd8c21-f7a6-47fb-adda-1f4d03b37c64\") " pod="openshift-dns/node-resolver-zq7r8" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309059 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-slash\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309097 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-run-openvswitch\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309122 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-run-netns\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309130 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-var-lib-cni-bin\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309145 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-var-lib-openvswitch\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309159 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-run-openvswitch\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309173 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pclt6\" (UniqueName: \"kubernetes.io/projected/214b45ed-9228-4589-8ecf-7cb43abf92a3-kube-api-access-pclt6\") pod \"iptables-alerter-qfn58\" (UID: \"214b45ed-9228-4589-8ecf-7cb43abf92a3\") " pod="openshift-network-operator/iptables-alerter-qfn58" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309198 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/04ea9897-2b15-4a68-93cf-cad10b6c7245-kubelet-config\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:44.309320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309221 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-run-netns\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309221 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-sysctl-conf\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309251 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-sysctl-d\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309272 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgbjf\" (UniqueName: \"kubernetes.io/projected/eec37bd8-a2d8-4d88-ade6-428188535543-kube-api-access-mgbjf\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309305 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-multus-socket-dir-parent\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309327 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-var-lib-kubelet\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309363 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-log-socket\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309379 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-var-lib-openvswitch\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309386 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c55e28b-e913-482a-b2ef-e9bf7d2e78e8-serviceca\") pod \"node-ca-vzjh4\" (UID: \"5c55e28b-e913-482a-b2ef-e9bf7d2e78e8\") " pod="openshift-image-registry/node-ca-vzjh4" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309412 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd7848c5-9bec-441f-bda7-a249bd472e1b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309421 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-sysctl-d\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309435 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-cnibin\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309339 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-sysctl-conf\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309435 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8f448e18-ba99-4279-b13a-b86003e4e2df-env-overrides\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309450 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/04ea9897-2b15-4a68-93cf-cad10b6c7245-kubelet-config\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309489 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-var-lib-cni-multus\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309496 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-multus-socket-dir-parent\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309198 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-slash\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.310145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309513 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-cni-bin\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309490 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-var-lib-kubelet\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309553 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-cni-bin\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309554 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-var-lib-cni-multus\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309581 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd7848c5-9bec-441f-bda7-a249bd472e1b-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309593 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8f448e18-ba99-4279-b13a-b86003e4e2df-ovn-node-metrics-cert\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309600 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-cni-binary-copy\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309687 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd7848c5-9bec-441f-bda7-a249bd472e1b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:44.309690 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309710 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48bd8c21-f7a6-47fb-adda-1f4d03b37c64-tmp-dir\") pod \"node-resolver-zq7r8\" (UID: \"48bd8c21-f7a6-47fb-adda-1f4d03b37c64\") " pod="openshift-dns/node-resolver-zq7r8" Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309714 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-etc-selinux\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309746 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tx6g\" (UniqueName: \"kubernetes.io/projected/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-kube-api-access-4tx6g\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309761 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-log-socket\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:44.309776 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret podName:04ea9897-2b15-4a68-93cf-cad10b6c7245 nodeName:}" failed. No retries permitted until 2026-06-15 03:08:44.809748121 +0000 UTC m=+2.966362323 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret") pod "global-pull-secret-syncer-66qb8" (UID: "04ea9897-2b15-4a68-93cf-cad10b6c7245") : object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309804 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eec37bd8-a2d8-4d88-ade6-428188535543-tmp\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309817 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-cnibin\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309829 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd7848c5-9bec-441f-bda7-a249bd472e1b-system-cni-dir\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.310972 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309855 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-os-release\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309870 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309880 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65pqv\" (UniqueName: \"kubernetes.io/projected/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-kube-api-access-65pqv\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309902 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/48bd8c21-f7a6-47fb-adda-1f4d03b37c64-hosts-file\") pod \"node-resolver-zq7r8\" (UID: \"48bd8c21-f7a6-47fb-adda-1f4d03b37c64\") " pod="openshift-dns/node-resolver-zq7r8" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309903 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd7848c5-9bec-441f-bda7-a249bd472e1b-system-cni-dir\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309927 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-os-release\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309936 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-kubelet\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309959 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/48bd8c21-f7a6-47fb-adda-1f4d03b37c64-hosts-file\") pod \"node-resolver-zq7r8\" (UID: \"48bd8c21-f7a6-47fb-adda-1f4d03b37c64\") " pod="openshift-dns/node-resolver-zq7r8" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309961 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9htqq\" (UniqueName: \"kubernetes.io/projected/8f448e18-ba99-4279-b13a-b86003e4e2df-kube-api-access-9htqq\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.309989 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/214b45ed-9228-4589-8ecf-7cb43abf92a3-host-slash\") pod \"iptables-alerter-qfn58\" (UID: \"214b45ed-9228-4589-8ecf-7cb43abf92a3\") " pod="openshift-network-operator/iptables-alerter-qfn58" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310015 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-run-multus-certs\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310039 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-etc-kubernetes\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310063 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-run-ovn\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310114 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/214b45ed-9228-4589-8ecf-7cb43abf92a3-iptables-alerter-script\") pod \"iptables-alerter-qfn58\" (UID: \"214b45ed-9228-4589-8ecf-7cb43abf92a3\") " pod="openshift-network-operator/iptables-alerter-qfn58" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-socket-dir\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310171 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-run-multus-certs\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310183 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-sys-fs\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310215 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-run-ovn\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.311779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310221 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-etc-kubernetes\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310248 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bd7848c5-9bec-441f-bda7-a249bd472e1b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310261 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-kubelet\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310271 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-sys\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310303 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-systemd\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310349 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-sys\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310347 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-multus-daemon-config\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310393 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1254732-a9f4-4953-80bc-4e01078878ac-konnectivity-ca\") pod \"konnectivity-agent-gc7b7\" (UID: \"b1254732-a9f4-4953-80bc-4e01078878ac\") " pod="kube-system/konnectivity-agent-gc7b7" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310352 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-systemd\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310429 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-device-dir\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310458 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bd7848c5-9bec-441f-bda7-a249bd472e1b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310484 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-run-k8s-cni-cncf-io\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310507 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-run-ovn-kubernetes\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310532 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310552 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c55e28b-e913-482a-b2ef-e9bf7d2e78e8-host\") pod \"node-ca-vzjh4\" (UID: \"5c55e28b-e913-482a-b2ef-e9bf7d2e78e8\") " pod="openshift-image-registry/node-ca-vzjh4" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310567 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-kubernetes\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310583 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eec37bd8-a2d8-4d88-ade6-428188535543-etc-tuned\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.312634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310586 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-host-run-k8s-cni-cncf-io\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310629 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310637 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-run-ovn-kubernetes\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310686 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-kubernetes\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310597 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-run-netns\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310726 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ln9v\" (UniqueName: \"kubernetes.io/projected/5c55e28b-e913-482a-b2ef-e9bf7d2e78e8-kube-api-access-6ln9v\") pod \"node-ca-vzjh4\" (UID: \"5c55e28b-e913-482a-b2ef-e9bf7d2e78e8\") " pod="openshift-image-registry/node-ca-vzjh4" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310755 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdczb\" (UniqueName: \"kubernetes.io/projected/bd7848c5-9bec-441f-bda7-a249bd472e1b-kube-api-access-tdczb\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310788 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bd7848c5-9bec-441f-bda7-a249bd472e1b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310789 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-system-cni-dir\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310829 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-system-cni-dir\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310831 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-multus-daemon-config\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310837 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-multus-cni-dir\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310889 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-multus-cni-dir\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310926 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-multus-conf-dir\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310942 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-run-netns\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310972 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bd7848c5-9bec-441f-bda7-a249bd472e1b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.310978 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-systemd-units\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.313689 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311009 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-multus-conf-dir\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311057 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-systemd-units\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311103 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-cni-netd\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311132 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-host-cni-netd\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311165 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/04ea9897-2b15-4a68-93cf-cad10b6c7245-dbus\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311196 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-modprobe-d\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311237 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/04ea9897-2b15-4a68-93cf-cad10b6c7245-dbus\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-sysconfig\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311312 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-sysconfig\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311322 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-run-systemd\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311352 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8f448e18-ba99-4279-b13a-b86003e4e2df-ovnkube-config\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311370 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eec37bd8-a2d8-4d88-ade6-428188535543-etc-modprobe-d\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311382 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8f448e18-ba99-4279-b13a-b86003e4e2df-run-systemd\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311419 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8f448e18-ba99-4279-b13a-b86003e4e2df-ovnkube-script-lib\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311445 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rc4w\" (UniqueName: \"kubernetes.io/projected/ff14c0e9-ddf7-4720-ac87-54a88354948e-kube-api-access-5rc4w\") pod \"network-metrics-daemon-9z4f4\" (UID: \"ff14c0e9-ddf7-4720-ac87-54a88354948e\") " pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311472 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w55f8\" (UniqueName: \"kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8\") pod \"network-check-target-zm8j4\" (UID: \"249c7536-df2e-4600-b69b-116d1432290a\") " pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311537 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd7848c5-9bec-441f-bda7-a249bd472e1b-cnibin\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.314561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311565 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpdqr\" (UniqueName: \"kubernetes.io/projected/48bd8c21-f7a6-47fb-adda-1f4d03b37c64-kube-api-access-xpdqr\") pod \"node-resolver-zq7r8\" (UID: \"48bd8c21-f7a6-47fb-adda-1f4d03b37c64\") " pod="openshift-dns/node-resolver-zq7r8" Jun 15 03:08:44.315368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311589 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs\") pod \"network-metrics-daemon-9z4f4\" (UID: \"ff14c0e9-ddf7-4720-ac87-54a88354948e\") " pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:44.315368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311617 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1254732-a9f4-4953-80bc-4e01078878ac-agent-certs\") pod \"konnectivity-agent-gc7b7\" (UID: \"b1254732-a9f4-4953-80bc-4e01078878ac\") " pod="kube-system/konnectivity-agent-gc7b7" Jun 15 03:08:44.315368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311641 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.315368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311667 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-registration-dir\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.315368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311748 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd7848c5-9bec-441f-bda7-a249bd472e1b-cnibin\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.315368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.311853 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8f448e18-ba99-4279-b13a-b86003e4e2df-ovnkube-config\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.315368 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:44.311938 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:44.315368 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:44.311992 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs podName:ff14c0e9-ddf7-4720-ac87-54a88354948e nodeName:}" failed. No retries permitted until 2026-06-15 03:08:44.811978482 +0000 UTC m=+2.968592683 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs") pod "network-metrics-daemon-9z4f4" (UID: "ff14c0e9-ddf7-4720-ac87-54a88354948e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:44.315368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.312143 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8f448e18-ba99-4279-b13a-b86003e4e2df-ovnkube-script-lib\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.315368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.313368 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eec37bd8-a2d8-4d88-ade6-428188535543-etc-tuned\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.315368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.313395 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eec37bd8-a2d8-4d88-ade6-428188535543-tmp\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.315368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.313522 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8f448e18-ba99-4279-b13a-b86003e4e2df-ovn-node-metrics-cert\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.320721 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.320382 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgbjf\" (UniqueName: \"kubernetes.io/projected/eec37bd8-a2d8-4d88-ade6-428188535543-kube-api-access-mgbjf\") pod \"tuned-d5x4c\" (UID: \"eec37bd8-a2d8-4d88-ade6-428188535543\") " pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.321447 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.321426 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpdqr\" (UniqueName: \"kubernetes.io/projected/48bd8c21-f7a6-47fb-adda-1f4d03b37c64-kube-api-access-xpdqr\") pod \"node-resolver-zq7r8\" (UID: \"48bd8c21-f7a6-47fb-adda-1f4d03b37c64\") " pod="openshift-dns/node-resolver-zq7r8" Jun 15 03:08:44.322365 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.322345 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65pqv\" (UniqueName: \"kubernetes.io/projected/df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f-kube-api-access-65pqv\") pod \"multus-bz9qb\" (UID: \"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f\") " pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.322504 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.322430 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9htqq\" (UniqueName: \"kubernetes.io/projected/8f448e18-ba99-4279-b13a-b86003e4e2df-kube-api-access-9htqq\") pod \"ovnkube-node-vsz59\" (UID: \"8f448e18-ba99-4279-b13a-b86003e4e2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.323211 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.322822 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rc4w\" (UniqueName: \"kubernetes.io/projected/ff14c0e9-ddf7-4720-ac87-54a88354948e-kube-api-access-5rc4w\") pod \"network-metrics-daemon-9z4f4\" (UID: \"ff14c0e9-ddf7-4720-ac87-54a88354948e\") " pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:44.324276 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.324254 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdczb\" (UniqueName: \"kubernetes.io/projected/bd7848c5-9bec-441f-bda7-a249bd472e1b-kube-api-access-tdczb\") pod \"multus-additional-cni-plugins-rzn4l\" (UID: \"bd7848c5-9bec-441f-bda7-a249bd472e1b\") " pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.412117 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412006 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pclt6\" (UniqueName: \"kubernetes.io/projected/214b45ed-9228-4589-8ecf-7cb43abf92a3-kube-api-access-pclt6\") pod \"iptables-alerter-qfn58\" (UID: \"214b45ed-9228-4589-8ecf-7cb43abf92a3\") " pod="openshift-network-operator/iptables-alerter-qfn58" Jun 15 03:08:44.412117 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412055 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c55e28b-e913-482a-b2ef-e9bf7d2e78e8-serviceca\") pod \"node-ca-vzjh4\" (UID: \"5c55e28b-e913-482a-b2ef-e9bf7d2e78e8\") " pod="openshift-image-registry/node-ca-vzjh4" Jun 15 03:08:44.412117 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412100 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-etc-selinux\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.412357 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412123 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tx6g\" (UniqueName: \"kubernetes.io/projected/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-kube-api-access-4tx6g\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.412357 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412151 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/214b45ed-9228-4589-8ecf-7cb43abf92a3-host-slash\") pod \"iptables-alerter-qfn58\" (UID: \"214b45ed-9228-4589-8ecf-7cb43abf92a3\") " pod="openshift-network-operator/iptables-alerter-qfn58" Jun 15 03:08:44.412357 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412175 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/214b45ed-9228-4589-8ecf-7cb43abf92a3-iptables-alerter-script\") pod \"iptables-alerter-qfn58\" (UID: \"214b45ed-9228-4589-8ecf-7cb43abf92a3\") " pod="openshift-network-operator/iptables-alerter-qfn58" Jun 15 03:08:44.412357 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412194 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-socket-dir\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.412357 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412213 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-sys-fs\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.412357 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412235 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1254732-a9f4-4953-80bc-4e01078878ac-konnectivity-ca\") pod \"konnectivity-agent-gc7b7\" (UID: \"b1254732-a9f4-4953-80bc-4e01078878ac\") " pod="kube-system/konnectivity-agent-gc7b7" Jun 15 03:08:44.412357 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412255 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-device-dir\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.412357 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412280 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c55e28b-e913-482a-b2ef-e9bf7d2e78e8-host\") pod \"node-ca-vzjh4\" (UID: \"5c55e28b-e913-482a-b2ef-e9bf7d2e78e8\") " pod="openshift-image-registry/node-ca-vzjh4" Jun 15 03:08:44.412357 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412303 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ln9v\" (UniqueName: \"kubernetes.io/projected/5c55e28b-e913-482a-b2ef-e9bf7d2e78e8-kube-api-access-6ln9v\") pod \"node-ca-vzjh4\" (UID: \"5c55e28b-e913-482a-b2ef-e9bf7d2e78e8\") " pod="openshift-image-registry/node-ca-vzjh4" Jun 15 03:08:44.412357 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412338 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w55f8\" (UniqueName: \"kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8\") pod \"network-check-target-zm8j4\" (UID: \"249c7536-df2e-4600-b69b-116d1432290a\") " pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:08:44.412692 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412409 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-device-dir\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.412692 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412440 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-sys-fs\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.412692 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412454 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-etc-selinux\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.412692 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412494 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c55e28b-e913-482a-b2ef-e9bf7d2e78e8-host\") pod \"node-ca-vzjh4\" (UID: \"5c55e28b-e913-482a-b2ef-e9bf7d2e78e8\") " pod="openshift-image-registry/node-ca-vzjh4" Jun 15 03:08:44.412692 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412515 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-socket-dir\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.412859 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412771 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/214b45ed-9228-4589-8ecf-7cb43abf92a3-host-slash\") pod \"iptables-alerter-qfn58\" (UID: \"214b45ed-9228-4589-8ecf-7cb43abf92a3\") " pod="openshift-network-operator/iptables-alerter-qfn58" Jun 15 03:08:44.412859 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412832 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1254732-a9f4-4953-80bc-4e01078878ac-agent-certs\") pod \"konnectivity-agent-gc7b7\" (UID: \"b1254732-a9f4-4953-80bc-4e01078878ac\") " pod="kube-system/konnectivity-agent-gc7b7" Jun 15 03:08:44.412932 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412861 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.412932 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412899 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-registration-dir\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.413029 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412949 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c55e28b-e913-482a-b2ef-e9bf7d2e78e8-serviceca\") pod \"node-ca-vzjh4\" (UID: \"5c55e28b-e913-482a-b2ef-e9bf7d2e78e8\") " pod="openshift-image-registry/node-ca-vzjh4" Jun 15 03:08:44.413029 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412973 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/214b45ed-9228-4589-8ecf-7cb43abf92a3-iptables-alerter-script\") pod \"iptables-alerter-qfn58\" (UID: \"214b45ed-9228-4589-8ecf-7cb43abf92a3\") " pod="openshift-network-operator/iptables-alerter-qfn58" Jun 15 03:08:44.413029 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412989 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.413029 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.412981 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-registration-dir\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.413244 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.413034 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1254732-a9f4-4953-80bc-4e01078878ac-konnectivity-ca\") pod \"konnectivity-agent-gc7b7\" (UID: \"b1254732-a9f4-4953-80bc-4e01078878ac\") " pod="kube-system/konnectivity-agent-gc7b7" Jun 15 03:08:44.415708 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.415682 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1254732-a9f4-4953-80bc-4e01078878ac-agent-certs\") pod \"konnectivity-agent-gc7b7\" (UID: \"b1254732-a9f4-4953-80bc-4e01078878ac\") " pod="kube-system/konnectivity-agent-gc7b7" Jun 15 03:08:44.417836 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:44.417813 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 03:08:44.417836 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:44.417838 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 03:08:44.417986 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:44.417851 2560 projected.go:196] Error preparing data for projected volume kube-api-access-w55f8 for pod openshift-network-diagnostics/network-check-target-zm8j4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:44.417986 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:44.417917 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8 podName:249c7536-df2e-4600-b69b-116d1432290a nodeName:}" failed. No retries permitted until 2026-06-15 03:08:44.917899734 +0000 UTC m=+3.074513941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w55f8" (UniqueName: "kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8") pod "network-check-target-zm8j4" (UID: "249c7536-df2e-4600-b69b-116d1432290a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:44.420416 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.420371 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ln9v\" (UniqueName: \"kubernetes.io/projected/5c55e28b-e913-482a-b2ef-e9bf7d2e78e8-kube-api-access-6ln9v\") pod \"node-ca-vzjh4\" (UID: \"5c55e28b-e913-482a-b2ef-e9bf7d2e78e8\") " pod="openshift-image-registry/node-ca-vzjh4" Jun 15 03:08:44.421408 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.421388 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tx6g\" (UniqueName: \"kubernetes.io/projected/7564b1a1-4be3-4701-bcb3-2f033f1e07ba-kube-api-access-4tx6g\") pod \"aws-ebs-csi-driver-node-xpnbt\" (UID: \"7564b1a1-4be3-4701-bcb3-2f033f1e07ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.421883 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.421860 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pclt6\" (UniqueName: \"kubernetes.io/projected/214b45ed-9228-4589-8ecf-7cb43abf92a3-kube-api-access-pclt6\") pod \"iptables-alerter-qfn58\" (UID: \"214b45ed-9228-4589-8ecf-7cb43abf92a3\") " pod="openshift-network-operator/iptables-alerter-qfn58" Jun 15 03:08:44.504974 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.504937 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zq7r8" Jun 15 03:08:44.513896 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.513873 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" Jun 15 03:08:44.522408 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.522390 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzn4l" Jun 15 03:08:44.528139 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.528117 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bz9qb" Jun 15 03:08:44.534977 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.534958 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:08:44.540804 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.540784 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vzjh4" Jun 15 03:08:44.546571 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.546554 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gc7b7" Jun 15 03:08:44.554023 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.554003 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qfn58" Jun 15 03:08:44.558565 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.558547 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" Jun 15 03:08:44.816319 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.816208 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:44.816481 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:44.816325 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs\") pod \"network-metrics-daemon-9z4f4\" (UID: \"ff14c0e9-ddf7-4720-ac87-54a88354948e\") " pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:44.816481 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:44.816378 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:44.816481 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:44.816434 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:44.816481 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:44.816459 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret podName:04ea9897-2b15-4a68-93cf-cad10b6c7245 nodeName:}" failed. No retries permitted until 2026-06-15 03:08:45.816437479 +0000 UTC m=+3.973051685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret") pod "global-pull-secret-syncer-66qb8" (UID: "04ea9897-2b15-4a68-93cf-cad10b6c7245") : object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:44.816700 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:44.816486 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs podName:ff14c0e9-ddf7-4720-ac87-54a88354948e nodeName:}" failed. No retries permitted until 2026-06-15 03:08:45.816470228 +0000 UTC m=+3.973084433 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs") pod "network-metrics-daemon-9z4f4" (UID: "ff14c0e9-ddf7-4720-ac87-54a88354948e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:45.017210 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:45.017172 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w55f8\" (UniqueName: \"kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8\") pod \"network-check-target-zm8j4\" (UID: \"249c7536-df2e-4600-b69b-116d1432290a\") " pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:08:45.017378 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:45.017356 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 03:08:45.017426 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:45.017382 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 03:08:45.017426 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:45.017392 2560 projected.go:196] Error preparing data for projected volume kube-api-access-w55f8 for pod openshift-network-diagnostics/network-check-target-zm8j4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:45.017505 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:45.017443 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8 podName:249c7536-df2e-4600-b69b-116d1432290a nodeName:}" failed. No retries permitted until 2026-06-15 03:08:46.017430187 +0000 UTC m=+4.174044373 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w55f8" (UniqueName: "kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8") pod "network-check-target-zm8j4" (UID: "249c7536-df2e-4600-b69b-116d1432290a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:45.238438 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:45.238394 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-14 03:03:44 +0000 UTC" deadline="2027-12-15 23:29:59.262685596 +0000 UTC" Jun 15 03:08:45.238694 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:45.238437 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13172h21m14.024250726s" Jun 15 03:08:45.256667 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:45.256614 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd7848c5_9bec_441f_bda7_a249bd472e1b.slice/crio-055bfa90305ad6142aeddd715cb114a8a23d2706a5c9b023e2c9475d25ba6043 WatchSource:0}: Error finding container 055bfa90305ad6142aeddd715cb114a8a23d2706a5c9b023e2c9475d25ba6043: Status 404 returned error can't find the container with id 055bfa90305ad6142aeddd715cb114a8a23d2706a5c9b023e2c9475d25ba6043 Jun 15 03:08:45.262276 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:45.262254 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:45.262409 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:45.262389 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:08:45.262923 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:45.262896 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf0dc0ff_b7d7_4d79_ad9d_ee4d600e157f.slice/crio-0a1f89c552808d936686685bd26ef15112575b104ca7d3eed69fc7bbc1fd81c7 WatchSource:0}: Error finding container 0a1f89c552808d936686685bd26ef15112575b104ca7d3eed69fc7bbc1fd81c7: Status 404 returned error can't find the container with id 0a1f89c552808d936686685bd26ef15112575b104ca7d3eed69fc7bbc1fd81c7 Jun 15 03:08:45.264287 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:45.264263 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeec37bd8_a2d8_4d88_ade6_428188535543.slice/crio-882ae9d19d4f2d9152d510a73ddfe0734976f5ecaf299a5b78cd49b81fcf4e14 WatchSource:0}: Error finding container 882ae9d19d4f2d9152d510a73ddfe0734976f5ecaf299a5b78cd49b81fcf4e14: Status 404 returned error can't find the container with id 882ae9d19d4f2d9152d510a73ddfe0734976f5ecaf299a5b78cd49b81fcf4e14 Jun 15 03:08:45.265037 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:45.265007 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48bd8c21_f7a6_47fb_adda_1f4d03b37c64.slice/crio-964cd28fe5ff19833d5a113b2f00cfa2698b01140c268110440486805833ae15 WatchSource:0}: Error finding container 964cd28fe5ff19833d5a113b2f00cfa2698b01140c268110440486805833ae15: Status 404 returned error can't find the container with id 964cd28fe5ff19833d5a113b2f00cfa2698b01140c268110440486805833ae15 Jun 15 03:08:45.265939 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:45.265874 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f448e18_ba99_4279_b13a_b86003e4e2df.slice/crio-563e9f536938c26a651c080440874ddff96690d1c76820e4a5c0967790ee4b69 WatchSource:0}: Error finding container 563e9f536938c26a651c080440874ddff96690d1c76820e4a5c0967790ee4b69: Status 404 returned error can't find the container with id 563e9f536938c26a651c080440874ddff96690d1c76820e4a5c0967790ee4b69 Jun 15 03:08:45.268334 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:45.268318 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c55e28b_e913_482a_b2ef_e9bf7d2e78e8.slice/crio-8435f391e01902719b00842b7d7724bb1fce69bc4bb71a18c572c5c98116cfb0 WatchSource:0}: Error finding container 8435f391e01902719b00842b7d7724bb1fce69bc4bb71a18c572c5c98116cfb0: Status 404 returned error can't find the container with id 8435f391e01902719b00842b7d7724bb1fce69bc4bb71a18c572c5c98116cfb0 Jun 15 03:08:45.272502 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:45.272479 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod214b45ed_9228_4589_8ecf_7cb43abf92a3.slice/crio-b2a091077acfbedc4598f95648aa0829555d625587aab4e488a0e40a289bee0e WatchSource:0}: Error finding container b2a091077acfbedc4598f95648aa0829555d625587aab4e488a0e40a289bee0e: Status 404 returned error can't find the container with id b2a091077acfbedc4598f95648aa0829555d625587aab4e488a0e40a289bee0e Jun 15 03:08:45.273736 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:08:45.273715 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7564b1a1_4be3_4701_bcb3_2f033f1e07ba.slice/crio-dea24564850d27af120ce20ff16767eca3bbbc6245c69925564d1edab59a1424 WatchSource:0}: Error finding container dea24564850d27af120ce20ff16767eca3bbbc6245c69925564d1edab59a1424: Status 404 returned error can't find the container with id dea24564850d27af120ce20ff16767eca3bbbc6245c69925564d1edab59a1424 Jun 15 03:08:45.275845 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:45.275823 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bz9qb" event={"ID":"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f","Type":"ContainerStarted","Data":"0a1f89c552808d936686685bd26ef15112575b104ca7d3eed69fc7bbc1fd81c7"} Jun 15 03:08:45.277742 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:45.277719 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzn4l" event={"ID":"bd7848c5-9bec-441f-bda7-a249bd472e1b","Type":"ContainerStarted","Data":"055bfa90305ad6142aeddd715cb114a8a23d2706a5c9b023e2c9475d25ba6043"} Jun 15 03:08:45.278784 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:45.278764 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vzjh4" event={"ID":"5c55e28b-e913-482a-b2ef-e9bf7d2e78e8","Type":"ContainerStarted","Data":"8435f391e01902719b00842b7d7724bb1fce69bc4bb71a18c572c5c98116cfb0"} Jun 15 03:08:45.280140 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:45.280016 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" event={"ID":"8f448e18-ba99-4279-b13a-b86003e4e2df","Type":"ContainerStarted","Data":"563e9f536938c26a651c080440874ddff96690d1c76820e4a5c0967790ee4b69"} Jun 15 03:08:45.281233 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:45.281200 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zq7r8" event={"ID":"48bd8c21-f7a6-47fb-adda-1f4d03b37c64","Type":"ContainerStarted","Data":"964cd28fe5ff19833d5a113b2f00cfa2698b01140c268110440486805833ae15"} Jun 15 03:08:45.282396 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:45.282358 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" event={"ID":"eec37bd8-a2d8-4d88-ade6-428188535543","Type":"ContainerStarted","Data":"882ae9d19d4f2d9152d510a73ddfe0734976f5ecaf299a5b78cd49b81fcf4e14"} Jun 15 03:08:45.822396 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:45.822310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs\") pod \"network-metrics-daemon-9z4f4\" (UID: \"ff14c0e9-ddf7-4720-ac87-54a88354948e\") " pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:45.822396 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:45.822376 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:45.822614 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:45.822496 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:45.822614 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:45.822499 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:45.822614 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:45.822559 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret podName:04ea9897-2b15-4a68-93cf-cad10b6c7245 nodeName:}" failed. No retries permitted until 2026-06-15 03:08:47.822540852 +0000 UTC m=+5.979155052 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret") pod "global-pull-secret-syncer-66qb8" (UID: "04ea9897-2b15-4a68-93cf-cad10b6c7245") : object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:45.822614 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:45.822578 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs podName:ff14c0e9-ddf7-4720-ac87-54a88354948e nodeName:}" failed. No retries permitted until 2026-06-15 03:08:47.822567963 +0000 UTC m=+5.979182154 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs") pod "network-metrics-daemon-9z4f4" (UID: "ff14c0e9-ddf7-4720-ac87-54a88354948e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:46.024440 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:46.024290 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w55f8\" (UniqueName: \"kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8\") pod \"network-check-target-zm8j4\" (UID: \"249c7536-df2e-4600-b69b-116d1432290a\") " pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:08:46.024624 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:46.024500 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 03:08:46.024624 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:46.024522 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 03:08:46.024624 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:46.024534 2560 projected.go:196] Error preparing data for projected volume kube-api-access-w55f8 for pod openshift-network-diagnostics/network-check-target-zm8j4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:46.024624 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:46.024594 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8 podName:249c7536-df2e-4600-b69b-116d1432290a nodeName:}" failed. No retries permitted until 2026-06-15 03:08:48.024576656 +0000 UTC m=+6.181190847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w55f8" (UniqueName: "kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8") pod "network-check-target-zm8j4" (UID: "249c7536-df2e-4600-b69b-116d1432290a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:46.239247 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:46.239150 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-14 03:03:44 +0000 UTC" deadline="2027-12-01 16:37:21.564542535 +0000 UTC" Jun 15 03:08:46.239247 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:46.239190 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12829h28m35.325356741s" Jun 15 03:08:46.262664 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:46.261997 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:08:46.262664 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:46.262137 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:08:46.262664 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:46.262522 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:46.262664 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:46.262625 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:08:46.309049 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:46.309011 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gc7b7" event={"ID":"b1254732-a9f4-4953-80bc-4e01078878ac","Type":"ContainerStarted","Data":"053aadf642fb2ca013ab1d6c876265239a5f7780e41cfd1839d49d56ef385aa8"} Jun 15 03:08:46.314066 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:46.314040 2560 generic.go:356] "Generic (PLEG): container finished" podID="68d6905ca08151f8a81b89181a1d6d46" containerID="ad8c396862a45de71284179f6b5a04f1b144b68c9e71f5a4342daeb420050ce4" exitCode=0 Jun 15 03:08:46.314212 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:46.314141 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" event={"ID":"68d6905ca08151f8a81b89181a1d6d46","Type":"ContainerDied","Data":"ad8c396862a45de71284179f6b5a04f1b144b68c9e71f5a4342daeb420050ce4"} Jun 15 03:08:46.321309 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:46.321214 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" event={"ID":"7564b1a1-4be3-4701-bcb3-2f033f1e07ba","Type":"ContainerStarted","Data":"dea24564850d27af120ce20ff16767eca3bbbc6245c69925564d1edab59a1424"} Jun 15 03:08:46.336256 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:46.334334 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" event={"ID":"3ccc0e83fda6561eb368da988cc50ea9","Type":"ContainerStarted","Data":"f65d8ab8ccf75e0608fe8cee3f1c59e594a56849b788a515ba21e030a2ca20ad"} Jun 15 03:08:46.339502 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:46.338728 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qfn58" event={"ID":"214b45ed-9228-4589-8ecf-7cb43abf92a3","Type":"ContainerStarted","Data":"b2a091077acfbedc4598f95648aa0829555d625587aab4e488a0e40a289bee0e"} Jun 15 03:08:46.351535 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:46.350793 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" podStartSLOduration=3.3507760380000002 podStartE2EDuration="3.350776038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:08:46.350333804 +0000 UTC m=+4.506948012" watchObservedRunningTime="2026-06-15 03:08:46.350776038 +0000 UTC m=+4.507390246" Jun 15 03:08:47.261459 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:47.261431 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:47.262037 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:47.261560 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:08:47.356211 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:47.356168 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" event={"ID":"68d6905ca08151f8a81b89181a1d6d46","Type":"ContainerStarted","Data":"81ee4ceaa2edbdf2d69aeaf209355e8e01f59bdc6b377c4d798ab81a403b993e"} Jun 15 03:08:47.843439 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:47.843400 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:47.843611 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:47.843498 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs\") pod \"network-metrics-daemon-9z4f4\" (UID: \"ff14c0e9-ddf7-4720-ac87-54a88354948e\") " pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:47.843670 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:47.843622 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:47.843723 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:47.843683 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs podName:ff14c0e9-ddf7-4720-ac87-54a88354948e nodeName:}" failed. No retries permitted until 2026-06-15 03:08:51.843664241 +0000 UTC m=+10.000278448 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs") pod "network-metrics-daemon-9z4f4" (UID: "ff14c0e9-ddf7-4720-ac87-54a88354948e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:47.844118 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:47.844094 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:47.844210 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:47.844151 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret podName:04ea9897-2b15-4a68-93cf-cad10b6c7245 nodeName:}" failed. No retries permitted until 2026-06-15 03:08:51.844136662 +0000 UTC m=+10.000750850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret") pod "global-pull-secret-syncer-66qb8" (UID: "04ea9897-2b15-4a68-93cf-cad10b6c7245") : object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:48.045311 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:48.045255 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w55f8\" (UniqueName: \"kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8\") pod \"network-check-target-zm8j4\" (UID: \"249c7536-df2e-4600-b69b-116d1432290a\") " pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:08:48.045487 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:48.045448 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 03:08:48.045487 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:48.045471 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 03:08:48.045487 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:48.045483 2560 projected.go:196] Error preparing data for projected volume kube-api-access-w55f8 for pod openshift-network-diagnostics/network-check-target-zm8j4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:48.045652 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:48.045551 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8 podName:249c7536-df2e-4600-b69b-116d1432290a nodeName:}" failed. No retries permitted until 2026-06-15 03:08:52.045532803 +0000 UTC m=+10.202147003 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w55f8" (UniqueName: "kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8") pod "network-check-target-zm8j4" (UID: "249c7536-df2e-4600-b69b-116d1432290a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:48.262285 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:48.262203 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:48.262285 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:48.262223 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:08:48.262764 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:48.262345 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:08:48.262764 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:48.262753 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:08:49.261513 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:49.261473 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:49.261699 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:49.261618 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:08:50.265815 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:50.265781 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:08:50.266270 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:50.265837 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:50.266270 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:50.265941 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:08:50.266399 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:50.266371 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:08:51.261373 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:51.261281 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:51.261535 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:51.261413 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:08:51.877767 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:51.877734 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:51.878149 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:51.877833 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs\") pod \"network-metrics-daemon-9z4f4\" (UID: \"ff14c0e9-ddf7-4720-ac87-54a88354948e\") " pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:51.878149 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:51.877859 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:51.878149 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:51.877934 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:51.878149 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:51.877937 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret podName:04ea9897-2b15-4a68-93cf-cad10b6c7245 nodeName:}" failed. No retries permitted until 2026-06-15 03:08:59.877915341 +0000 UTC m=+18.034529545 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret") pod "global-pull-secret-syncer-66qb8" (UID: "04ea9897-2b15-4a68-93cf-cad10b6c7245") : object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:51.878149 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:51.877992 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs podName:ff14c0e9-ddf7-4720-ac87-54a88354948e nodeName:}" failed. No retries permitted until 2026-06-15 03:08:59.877975668 +0000 UTC m=+18.034589861 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs") pod "network-metrics-daemon-9z4f4" (UID: "ff14c0e9-ddf7-4720-ac87-54a88354948e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:52.079617 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:52.079577 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w55f8\" (UniqueName: \"kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8\") pod \"network-check-target-zm8j4\" (UID: \"249c7536-df2e-4600-b69b-116d1432290a\") " pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:08:52.079787 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:52.079739 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 03:08:52.079787 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:52.079756 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 03:08:52.079787 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:52.079769 2560 projected.go:196] Error preparing data for projected volume kube-api-access-w55f8 for pod openshift-network-diagnostics/network-check-target-zm8j4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:52.079939 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:52.079828 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8 podName:249c7536-df2e-4600-b69b-116d1432290a nodeName:}" failed. No retries permitted until 2026-06-15 03:09:00.079810688 +0000 UTC m=+18.236424890 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w55f8" (UniqueName: "kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8") pod "network-check-target-zm8j4" (UID: "249c7536-df2e-4600-b69b-116d1432290a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:52.262443 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:52.262179 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:08:52.262443 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:52.262279 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:52.262443 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:52.262304 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:08:52.262443 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:52.262359 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:08:53.261985 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:53.261491 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:53.261985 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:53.261636 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:08:54.261550 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:54.261516 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:54.261737 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:54.261648 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:08:54.261737 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:54.261516 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:08:54.261851 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:54.261746 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:08:55.262166 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:55.262126 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:55.262635 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:55.262272 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:08:56.262008 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:56.261535 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:56.262008 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:56.261658 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:08:56.262008 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:56.261735 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:08:56.262008 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:56.261821 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:08:57.261244 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:57.261207 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:57.261437 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:57.261339 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:08:58.261433 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:58.261391 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:58.261855 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:58.261537 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:08:58.261855 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:58.261574 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:08:58.261855 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:58.261696 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:08:59.261304 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:59.261267 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:59.261490 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:59.261404 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:08:59.942404 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:59.942206 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:08:59.942597 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:59.942359 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:59.942597 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:08:59.942478 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs\") pod \"network-metrics-daemon-9z4f4\" (UID: \"ff14c0e9-ddf7-4720-ac87-54a88354948e\") " pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:08:59.942597 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:59.942548 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret podName:04ea9897-2b15-4a68-93cf-cad10b6c7245 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:15.942523962 +0000 UTC m=+34.099138167 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret") pod "global-pull-secret-syncer-66qb8" (UID: "04ea9897-2b15-4a68-93cf-cad10b6c7245") : object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:59.942597 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:59.942582 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:59.942755 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:08:59.942631 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs podName:ff14c0e9-ddf7-4720-ac87-54a88354948e nodeName:}" failed. No retries permitted until 2026-06-15 03:09:15.942615094 +0000 UTC m=+34.099229281 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs") pod "network-metrics-daemon-9z4f4" (UID: "ff14c0e9-ddf7-4720-ac87-54a88354948e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:09:00.143952 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:00.143910 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w55f8\" (UniqueName: \"kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8\") pod \"network-check-target-zm8j4\" (UID: \"249c7536-df2e-4600-b69b-116d1432290a\") " pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:00.144167 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:00.144143 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 03:09:00.144228 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:00.144169 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 03:09:00.144228 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:00.144182 2560 projected.go:196] Error preparing data for projected volume kube-api-access-w55f8 for pod openshift-network-diagnostics/network-check-target-zm8j4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:09:00.144329 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:00.144251 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8 podName:249c7536-df2e-4600-b69b-116d1432290a nodeName:}" failed. No retries permitted until 2026-06-15 03:09:16.144232516 +0000 UTC m=+34.300846717 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w55f8" (UniqueName: "kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8") pod "network-check-target-zm8j4" (UID: "249c7536-df2e-4600-b69b-116d1432290a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:09:00.262349 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:00.262259 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:00.262766 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:00.262397 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:09:00.262766 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:00.262457 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:09:00.262766 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:00.262569 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:09:01.261974 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:01.261940 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:09:01.262181 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:01.262078 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:09:02.262850 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:02.262817 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:02.263330 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:02.262907 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:09:02.263330 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:02.262994 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:09:02.263330 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:02.263147 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:09:03.261447 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:03.261409 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:09:03.261645 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:03.261541 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:09:04.261586 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.261316 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:04.262252 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.261349 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:09:04.262252 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:04.261716 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:09:04.262252 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:04.261753 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:09:04.394856 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.394777 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:09:04.395163 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.395138 2560 generic.go:356] "Generic (PLEG): container finished" podID="8f448e18-ba99-4279-b13a-b86003e4e2df" containerID="c15bea6c3c714ff051fdd73a3efb224843a1c1f39b20f2ae4b57f074c4876a4e" exitCode=1 Jun 15 03:09:04.395225 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.395203 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" event={"ID":"8f448e18-ba99-4279-b13a-b86003e4e2df","Type":"ContainerStarted","Data":"c8c3ea9d84957412009e898acc749149b5dc3ecc77ee24ca2b50df1e3329f215"} Jun 15 03:09:04.395317 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.395300 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" event={"ID":"8f448e18-ba99-4279-b13a-b86003e4e2df","Type":"ContainerStarted","Data":"81497bf47515d441a84bd7d58c47932c4669a6865ae42536f820500890b1e6f1"} Jun 15 03:09:04.395427 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.395350 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" event={"ID":"8f448e18-ba99-4279-b13a-b86003e4e2df","Type":"ContainerStarted","Data":"859ed755d747e64ee324d74544198b87becfca509f7f0b9379d46f405067a392"} Jun 15 03:09:04.395427 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.395396 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" event={"ID":"8f448e18-ba99-4279-b13a-b86003e4e2df","Type":"ContainerDied","Data":"c15bea6c3c714ff051fdd73a3efb224843a1c1f39b20f2ae4b57f074c4876a4e"} Jun 15 03:09:04.395528 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.395447 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" event={"ID":"8f448e18-ba99-4279-b13a-b86003e4e2df","Type":"ContainerStarted","Data":"65fb5750e6aa19b513e933ed9cfdf71d8bc74d9ac5159331d6619e19f6087fd0"} Jun 15 03:09:04.396660 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.396639 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bz9qb" event={"ID":"df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f","Type":"ContainerStarted","Data":"e0cb9feddc6b7b10c864e1ef7a11e1e448b7851a23d2cca1e74abbb883c3df30"} Jun 15 03:09:04.398458 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.398432 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gc7b7" event={"ID":"b1254732-a9f4-4953-80bc-4e01078878ac","Type":"ContainerStarted","Data":"a9e6e76ab9c875260b7c9313287977b8e5e189d985e547ed681af3988b2ef360"} Jun 15 03:09:04.399939 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.399913 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vzjh4" event={"ID":"5c55e28b-e913-482a-b2ef-e9bf7d2e78e8","Type":"ContainerStarted","Data":"8ef934d36117011748599d68530bccbb9a64d7ec368724b872f878593d2b50bb"} Jun 15 03:09:04.401384 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.401365 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zq7r8" event={"ID":"48bd8c21-f7a6-47fb-adda-1f4d03b37c64","Type":"ContainerStarted","Data":"84746d6ddc7aba06cf90bc3326e2e47d8ba569ee446f1a1f53a89ddd5f2e2833"} Jun 15 03:09:04.403069 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.403042 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" event={"ID":"eec37bd8-a2d8-4d88-ade6-428188535543","Type":"ContainerStarted","Data":"0bdba40b2b1de587a580034124943124d807910c3b562dbecf181db9c444091d"} Jun 15 03:09:04.404774 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.404747 2560 generic.go:356] "Generic (PLEG): container finished" podID="bd7848c5-9bec-441f-bda7-a249bd472e1b" containerID="fb44bbddc233f6be15d3ec91869988d9883e5eadad78af42648e71a5c7bfea44" exitCode=0 Jun 15 03:09:04.404875 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.404830 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzn4l" event={"ID":"bd7848c5-9bec-441f-bda7-a249bd472e1b","Type":"ContainerDied","Data":"fb44bbddc233f6be15d3ec91869988d9883e5eadad78af42648e71a5c7bfea44"} Jun 15 03:09:04.407036 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.406996 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" event={"ID":"7564b1a1-4be3-4701-bcb3-2f033f1e07ba","Type":"ContainerStarted","Data":"388b90f889b557a2bd98c2fde443f27e54e54f2077f9199baae423d20b303baf"} Jun 15 03:09:04.417666 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.417622 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" podStartSLOduration=21.417610197 podStartE2EDuration="21.417610197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:08:47.373116117 +0000 UTC m=+5.529730328" watchObservedRunningTime="2026-06-15 03:09:04.417610197 +0000 UTC m=+22.574224405" Jun 15 03:09:04.431407 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.431368 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bz9qb" podStartSLOduration=3.8944362139999997 podStartE2EDuration="22.431355843s" totalImagesPullingTime="18.536919629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:04.417240389 +0000 UTC m=+22.573854597" watchObservedRunningTime="2026-06-15 03:09:04.431355843 +0000 UTC m=+22.587970051" Jun 15 03:09:04.431782 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.431756 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vzjh4" podStartSLOduration=3.9342329940000003 podStartE2EDuration="22.431748552s" totalImagesPullingTime="18.497515558s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:04.431286243 +0000 UTC m=+22.587900451" watchObservedRunningTime="2026-06-15 03:09:04.431748552 +0000 UTC m=+22.588362754" Jun 15 03:09:04.445738 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.445707 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gc7b7" podStartSLOduration=12.316653732 podStartE2EDuration="22.445695814s" totalImagesPullingTime="10.129042082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:04.445505887 +0000 UTC m=+22.602120094" watchObservedRunningTime="2026-06-15 03:09:04.445695814 +0000 UTC m=+22.602310021" Jun 15 03:09:04.464296 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.464263 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-d5x4c" podStartSLOduration=3.960919514 podStartE2EDuration="22.464253397s" totalImagesPullingTime="18.503333883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:04.463728337 +0000 UTC m=+22.620342556" watchObservedRunningTime="2026-06-15 03:09:04.464253397 +0000 UTC m=+22.620867606" Jun 15 03:09:04.502521 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.502483 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zq7r8" podStartSLOduration=4.000902474 podStartE2EDuration="22.502469495s" totalImagesPullingTime="18.501567021s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:04.47868245 +0000 UTC m=+22.635296658" watchObservedRunningTime="2026-06-15 03:09:04.502469495 +0000 UTC m=+22.659083703" Jun 15 03:09:04.972735 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:04.972603 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 15 03:09:05.261909 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:05.261878 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:09:05.262263 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:05.261990 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:09:05.262926 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:05.262852 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-15T03:09:04.972729319Z","UUID":"92d53322-9029-40de-bbf6-3c61eaf09eaa","Handler":null,"Name":"","Endpoint":""} Jun 15 03:09:05.264358 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:05.264338 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 15 03:09:05.264482 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:05.264376 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 15 03:09:05.411909 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:05.411838 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:09:05.412233 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:05.412204 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" event={"ID":"8f448e18-ba99-4279-b13a-b86003e4e2df","Type":"ContainerStarted","Data":"f1860bf624297a0f65df60d1638836530c298dae3939a01944f0c3de32d33641"} Jun 15 03:09:05.413507 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:05.413488 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qfn58" event={"ID":"214b45ed-9228-4589-8ecf-7cb43abf92a3","Type":"ContainerStarted","Data":"e5e4682714cd33753deabc0e2787a3c39b8d262ab45ce5da7db9bfb7c3a84bf3"} Jun 15 03:09:05.414999 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:05.414980 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" event={"ID":"7564b1a1-4be3-4701-bcb3-2f033f1e07ba","Type":"ContainerStarted","Data":"ebe892291e73035e03b109105f5a776a770cd7d5e34d9af449bf61f8a384a2ad"} Jun 15 03:09:05.428029 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:05.427990 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qfn58" podStartSLOduration=4.935390181 podStartE2EDuration="23.427974375s" totalImagesPullingTime="18.492584194s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:05.427532198 +0000 UTC m=+23.584146407" watchObservedRunningTime="2026-06-15 03:09:05.427974375 +0000 UTC m=+23.584588584" Jun 15 03:09:06.262121 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:06.262003 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:09:06.262633 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:06.262152 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:09:06.262633 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:06.262208 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:06.262633 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:06.262318 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:09:06.419218 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:06.419179 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" event={"ID":"7564b1a1-4be3-4701-bcb3-2f033f1e07ba","Type":"ContainerStarted","Data":"04d35571371e340cb4fbdb0d276ec32cb7c204dfd3d83cac12b9985c8b12f45a"} Jun 15 03:09:06.437330 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:06.437278 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xpnbt" podStartSLOduration=4.229014799 podStartE2EDuration="24.437264353s" totalImagesPullingTime="20.208249554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:06.437049152 +0000 UTC m=+24.593663359" watchObservedRunningTime="2026-06-15 03:09:06.437264353 +0000 UTC m=+24.593878560" Jun 15 03:09:07.184225 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:07.184193 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gc7b7" Jun 15 03:09:07.184842 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:07.184815 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gc7b7" Jun 15 03:09:07.261690 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:07.261664 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:09:07.261829 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:07.261790 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:09:07.425141 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:07.425115 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:09:07.425753 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:07.425525 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" event={"ID":"8f448e18-ba99-4279-b13a-b86003e4e2df","Type":"ContainerStarted","Data":"adf9ab0a9535c34341f4592bd6163874ac261bccdc7a82c14004c4d3b2d066b7"} Jun 15 03:09:08.261259 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:08.261228 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:08.261439 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:08.261228 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:09:08.261439 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:08.261344 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:09:08.261556 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:08.261451 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:09:09.262283 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:09.262251 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:09:09.262842 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:09.262405 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:09:09.433555 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:09.433425 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:09:09.433934 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:09.433910 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" event={"ID":"8f448e18-ba99-4279-b13a-b86003e4e2df","Type":"ContainerStarted","Data":"da2fb16646789f717222b4d0cc9cfdecbe5aae1a63cefd8c99fb6fc9c4bdb3ed"} Jun 15 03:09:10.261520 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:10.261486 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:09:10.261673 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:10.261588 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:09:10.261734 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:10.261692 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:10.261812 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:10.261791 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:09:10.437875 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:10.437839 2560 generic.go:356] "Generic (PLEG): container finished" podID="bd7848c5-9bec-441f-bda7-a249bd472e1b" containerID="0f7ea9af8d7a4ee570a6e6174d108b3eac0467eaf3d51dc2f10bfb4c8619a55b" exitCode=0 Jun 15 03:09:10.438350 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:10.437885 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzn4l" event={"ID":"bd7848c5-9bec-441f-bda7-a249bd472e1b","Type":"ContainerDied","Data":"0f7ea9af8d7a4ee570a6e6174d108b3eac0467eaf3d51dc2f10bfb4c8619a55b"} Jun 15 03:09:10.438350 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:10.438176 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:09:10.438350 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:10.438227 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:09:10.438350 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:10.438264 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:09:10.438350 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:10.438314 2560 scope.go:117] "RemoveContainer" containerID="c15bea6c3c714ff051fdd73a3efb224843a1c1f39b20f2ae4b57f074c4876a4e" Jun 15 03:09:10.454491 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:10.454472 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:09:10.458506 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:10.458486 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:09:11.261861 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:11.261832 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:09:11.262075 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:11.261938 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:09:11.348581 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:11.348544 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-66qb8"] Jun 15 03:09:11.348734 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:11.348706 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:09:11.348813 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:11.348793 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:09:11.351500 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:11.351466 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zm8j4"] Jun 15 03:09:11.351637 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:11.351573 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:11.351707 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:11.351684 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:09:11.352305 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:11.352280 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9z4f4"] Jun 15 03:09:11.391321 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:11.391298 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gc7b7" Jun 15 03:09:11.391474 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:11.391396 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 15 03:09:11.391878 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:11.391854 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gc7b7" Jun 15 03:09:11.443863 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:11.443839 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:09:11.444295 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:11.444263 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" event={"ID":"8f448e18-ba99-4279-b13a-b86003e4e2df","Type":"ContainerStarted","Data":"61420961bf3b8c3e74c7a5c8c56566946b3d91952dbd31d790f8c8da48f96f69"} Jun 15 03:09:11.444413 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:11.444398 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:09:11.444566 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:11.444545 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:09:11.472767 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:11.472590 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" podStartSLOduration=10.908428834 podStartE2EDuration="29.472576878s" totalImagesPullingTime="18.564148044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:11.470988313 +0000 UTC m=+29.627602521" watchObservedRunningTime="2026-06-15 03:09:11.472576878 +0000 UTC m=+29.629191086" Jun 15 03:09:12.448940 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:12.448909 2560 generic.go:356] "Generic (PLEG): container finished" podID="bd7848c5-9bec-441f-bda7-a249bd472e1b" containerID="2f27a7d5039dc74fedc52d08f0c93f77220422075b60d75b7cfd155ff6b25ecb" exitCode=0 Jun 15 03:09:12.449361 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:12.448992 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzn4l" event={"ID":"bd7848c5-9bec-441f-bda7-a249bd472e1b","Type":"ContainerDied","Data":"2f27a7d5039dc74fedc52d08f0c93f77220422075b60d75b7cfd155ff6b25ecb"} Jun 15 03:09:13.262265 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:13.262233 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:13.262402 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:13.262249 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:09:13.262402 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:13.262350 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:09:13.262402 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:13.262358 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:09:13.262517 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:13.262428 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:09:13.262517 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:13.262490 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:09:13.454236 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:13.454210 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzn4l" event={"ID":"bd7848c5-9bec-441f-bda7-a249bd472e1b","Type":"ContainerStarted","Data":"f8f7ce1f9c61dd03c6374432cef5f5bc36440633b1c62a0a707936ae8016631c"} Jun 15 03:09:14.458979 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:14.458944 2560 generic.go:356] "Generic (PLEG): container finished" podID="bd7848c5-9bec-441f-bda7-a249bd472e1b" containerID="f8f7ce1f9c61dd03c6374432cef5f5bc36440633b1c62a0a707936ae8016631c" exitCode=0 Jun 15 03:09:14.459356 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:14.459015 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzn4l" event={"ID":"bd7848c5-9bec-441f-bda7-a249bd472e1b","Type":"ContainerDied","Data":"f8f7ce1f9c61dd03c6374432cef5f5bc36440633b1c62a0a707936ae8016631c"} Jun 15 03:09:15.261235 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:15.261206 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:09:15.261381 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:15.261206 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:09:15.261381 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:15.261309 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66qb8" podUID="04ea9897-2b15-4a68-93cf-cad10b6c7245" Jun 15 03:09:15.261498 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:15.261389 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:09:15.261498 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:15.261205 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:15.261498 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:15.261457 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zm8j4" podUID="249c7536-df2e-4600-b69b-116d1432290a" Jun 15 03:09:15.962001 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:15.961960 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:09:15.962487 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:15.962041 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs\") pod \"network-metrics-daemon-9z4f4\" (UID: \"ff14c0e9-ddf7-4720-ac87-54a88354948e\") " pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:09:15.962487 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:15.962143 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:09:15.962487 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:15.962200 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs podName:ff14c0e9-ddf7-4720-ac87-54a88354948e nodeName:}" failed. No retries permitted until 2026-06-15 03:09:47.962186079 +0000 UTC m=+66.118800270 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs") pod "network-metrics-daemon-9z4f4" (UID: "ff14c0e9-ddf7-4720-ac87-54a88354948e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:09:15.962487 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:15.962147 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 03:09:15.962487 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:15.962273 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret podName:04ea9897-2b15-4a68-93cf-cad10b6c7245 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:47.962261247 +0000 UTC m=+66.118875432 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret") pod "global-pull-secret-syncer-66qb8" (UID: "04ea9897-2b15-4a68-93cf-cad10b6c7245") : object "kube-system"/"original-pull-secret" not registered Jun 15 03:09:16.164457 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.164402 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w55f8\" (UniqueName: \"kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8\") pod \"network-check-target-zm8j4\" (UID: \"249c7536-df2e-4600-b69b-116d1432290a\") " pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:16.164702 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:16.164656 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 03:09:16.164774 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:16.164708 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 03:09:16.164774 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:16.164724 2560 projected.go:196] Error preparing data for projected volume kube-api-access-w55f8 for pod openshift-network-diagnostics/network-check-target-zm8j4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:09:16.164881 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:16.164788 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8 podName:249c7536-df2e-4600-b69b-116d1432290a nodeName:}" failed. No retries permitted until 2026-06-15 03:09:48.164769101 +0000 UTC m=+66.321383311 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w55f8" (UniqueName: "kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8") pod "network-check-target-zm8j4" (UID: "249c7536-df2e-4600-b69b-116d1432290a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:09:16.675039 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.675012 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeReady" Jun 15 03:09:16.675222 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.675176 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 15 03:09:16.712627 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.712547 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6786c7fbc5-8q824"] Jun 15 03:09:16.735857 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.735828 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5zmqd"] Jun 15 03:09:16.736040 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.735958 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.738653 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.738628 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4l6gg\"" Jun 15 03:09:16.738790 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.738628 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 15 03:09:16.738858 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.738789 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 15 03:09:16.738910 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.738874 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 15 03:09:16.745645 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.745620 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 15 03:09:16.753878 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.753840 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6786c7fbc5-8q824"] Jun 15 03:09:16.754006 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.753885 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:16.754006 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.753893 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s7jcv"] Jun 15 03:09:16.756712 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.756688 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 15 03:09:16.756820 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.756804 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b8jdx\"" Jun 15 03:09:16.756878 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.756804 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 15 03:09:16.767944 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.767916 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5zmqd"] Jun 15 03:09:16.768068 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.767961 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s7jcv"] Jun 15 03:09:16.768068 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.768034 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s7jcv" Jun 15 03:09:16.770620 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.770592 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 15 03:09:16.770805 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.770779 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 15 03:09:16.770909 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.770819 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dqgpx\"" Jun 15 03:09:16.770909 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.770884 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 15 03:09:16.870539 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.870500 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-tmp-dir\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:16.870727 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.870553 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50f26ba5-125d-4c08-bc95-0fef32a874f6-trusted-ca\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.870727 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.870589 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5-cert\") pod \"ingress-canary-s7jcv\" (UID: \"5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5\") " pod="openshift-ingress-canary/ingress-canary-s7jcv" Jun 15 03:09:16.870727 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.870684 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50f26ba5-125d-4c08-bc95-0fef32a874f6-ca-trust-extracted\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.870727 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.870721 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-config-volume\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:16.870920 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.870761 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50f26ba5-125d-4c08-bc95-0fef32a874f6-installation-pull-secrets\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.870920 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.870813 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:16.870920 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.870839 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.870920 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.870868 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-certificates\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.870920 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.870887 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k449\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-kube-api-access-5k449\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.870920 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.870907 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4npqg\" (UniqueName: \"kubernetes.io/projected/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-kube-api-access-4npqg\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:16.871190 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.870933 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-bound-sa-token\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.871190 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.870980 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2crj\" (UniqueName: \"kubernetes.io/projected/5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5-kube-api-access-m2crj\") pod \"ingress-canary-s7jcv\" (UID: \"5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5\") " pod="openshift-ingress-canary/ingress-canary-s7jcv" Jun 15 03:09:16.871190 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.871147 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50f26ba5-125d-4c08-bc95-0fef32a874f6-image-registry-private-configuration\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.971707 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.971621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2crj\" (UniqueName: \"kubernetes.io/projected/5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5-kube-api-access-m2crj\") pod \"ingress-canary-s7jcv\" (UID: \"5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5\") " pod="openshift-ingress-canary/ingress-canary-s7jcv" Jun 15 03:09:16.971707 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.971693 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50f26ba5-125d-4c08-bc95-0fef32a874f6-image-registry-private-configuration\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.972222 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.971724 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-tmp-dir\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:16.972222 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.971745 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50f26ba5-125d-4c08-bc95-0fef32a874f6-trusted-ca\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.972222 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.971761 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5-cert\") pod \"ingress-canary-s7jcv\" (UID: \"5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5\") " pod="openshift-ingress-canary/ingress-canary-s7jcv" Jun 15 03:09:16.972222 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.971794 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50f26ba5-125d-4c08-bc95-0fef32a874f6-ca-trust-extracted\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.972222 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.971817 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-config-volume\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:16.972222 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.971860 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50f26ba5-125d-4c08-bc95-0fef32a874f6-installation-pull-secrets\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.972222 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.971884 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:16.972222 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.971911 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.972222 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.971937 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-certificates\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.972222 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.971959 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k449\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-kube-api-access-5k449\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.972222 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.971983 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4npqg\" (UniqueName: \"kubernetes.io/projected/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-kube-api-access-4npqg\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:16.972222 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.972012 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-bound-sa-token\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.972222 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.972135 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-tmp-dir\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:16.972847 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:16.972238 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 03:09:16.972847 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:16.972253 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6786c7fbc5-8q824: secret "image-registry-tls" not found Jun 15 03:09:16.972847 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:16.972316 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls podName:50f26ba5-125d-4c08-bc95-0fef32a874f6 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:17.472296555 +0000 UTC m=+35.628910745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls") pod "image-registry-6786c7fbc5-8q824" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6") : secret "image-registry-tls" not found Jun 15 03:09:16.972847 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:16.972560 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:09:16.972847 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.972597 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-certificates\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.972847 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:16.972606 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls podName:f10bedca-8f8a-4db1-9b58-8d3d854ece5d nodeName:}" failed. No retries permitted until 2026-06-15 03:09:17.472593725 +0000 UTC m=+35.629207911 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls") pod "dns-default-5zmqd" (UID: "f10bedca-8f8a-4db1-9b58-8d3d854ece5d") : secret "dns-default-metrics-tls" not found Jun 15 03:09:16.972847 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.972792 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50f26ba5-125d-4c08-bc95-0fef32a874f6-ca-trust-extracted\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.972847 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.972812 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50f26ba5-125d-4c08-bc95-0fef32a874f6-trusted-ca\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.973238 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.973129 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-config-volume\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:16.976934 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.976807 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50f26ba5-125d-4c08-bc95-0fef32a874f6-installation-pull-secrets\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.977059 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.976812 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5-cert\") pod \"ingress-canary-s7jcv\" (UID: \"5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5\") " pod="openshift-ingress-canary/ingress-canary-s7jcv" Jun 15 03:09:16.977059 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.976836 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50f26ba5-125d-4c08-bc95-0fef32a874f6-image-registry-private-configuration\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.983839 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.983573 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4npqg\" (UniqueName: \"kubernetes.io/projected/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-kube-api-access-4npqg\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:16.983839 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.983763 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2crj\" (UniqueName: \"kubernetes.io/projected/5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5-kube-api-access-m2crj\") pod \"ingress-canary-s7jcv\" (UID: \"5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5\") " pod="openshift-ingress-canary/ingress-canary-s7jcv" Jun 15 03:09:16.984033 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.983860 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k449\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-kube-api-access-5k449\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:16.984310 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:16.984289 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-bound-sa-token\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:17.079027 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:17.078993 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s7jcv" Jun 15 03:09:17.261653 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:17.261550 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:17.261653 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:17.261576 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:09:17.261900 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:17.261663 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:09:17.265541 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:17.264977 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hxgvb\"" Jun 15 03:09:17.265541 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:17.265020 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 15 03:09:17.265541 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:17.265031 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 15 03:09:17.265541 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:17.265061 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 15 03:09:17.265541 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:17.265288 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 15 03:09:17.268147 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:17.267484 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lnpms\"" Jun 15 03:09:17.476618 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:17.476581 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:17.476848 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:17.476625 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:17.476848 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:17.476743 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 03:09:17.476848 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:17.476754 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6786c7fbc5-8q824: secret "image-registry-tls" not found Jun 15 03:09:17.476848 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:17.476742 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:09:17.476848 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:17.476804 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls podName:50f26ba5-125d-4c08-bc95-0fef32a874f6 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:18.476785793 +0000 UTC m=+36.633400052 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls") pod "image-registry-6786c7fbc5-8q824" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6") : secret "image-registry-tls" not found Jun 15 03:09:17.476848 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:17.476852 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls podName:f10bedca-8f8a-4db1-9b58-8d3d854ece5d nodeName:}" failed. No retries permitted until 2026-06-15 03:09:18.47683397 +0000 UTC m=+36.633448167 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls") pod "dns-default-5zmqd" (UID: "f10bedca-8f8a-4db1-9b58-8d3d854ece5d") : secret "dns-default-metrics-tls" not found Jun 15 03:09:17.957888 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:17.957845 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s7jcv"] Jun 15 03:09:18.035894 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:09:18.035858 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dab88de_1c68_4f1e_a4a3_e6a92f7b46c5.slice/crio-9a83bc7f885f64db7764df15860f9f4420c2d64fbfb79739d1b6767a2fc08bae WatchSource:0}: Error finding container 9a83bc7f885f64db7764df15860f9f4420c2d64fbfb79739d1b6767a2fc08bae: Status 404 returned error can't find the container with id 9a83bc7f885f64db7764df15860f9f4420c2d64fbfb79739d1b6767a2fc08bae Jun 15 03:09:18.471068 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:18.470987 2560 generic.go:356] "Generic (PLEG): container finished" podID="bd7848c5-9bec-441f-bda7-a249bd472e1b" containerID="c5ed73cd072f985d778ee86051fadd373de15f97b8a448e6bcf8fd566eb1c45d" exitCode=0 Jun 15 03:09:18.471283 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:18.471069 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzn4l" event={"ID":"bd7848c5-9bec-441f-bda7-a249bd472e1b","Type":"ContainerDied","Data":"c5ed73cd072f985d778ee86051fadd373de15f97b8a448e6bcf8fd566eb1c45d"} Jun 15 03:09:18.472220 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:18.472202 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s7jcv" event={"ID":"5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5","Type":"ContainerStarted","Data":"9a83bc7f885f64db7764df15860f9f4420c2d64fbfb79739d1b6767a2fc08bae"} Jun 15 03:09:18.486265 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:18.486242 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:18.486368 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:18.486271 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:18.486407 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:18.486389 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:09:18.486454 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:18.486443 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls podName:f10bedca-8f8a-4db1-9b58-8d3d854ece5d nodeName:}" failed. No retries permitted until 2026-06-15 03:09:20.486428571 +0000 UTC m=+38.643042761 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls") pod "dns-default-5zmqd" (UID: "f10bedca-8f8a-4db1-9b58-8d3d854ece5d") : secret "dns-default-metrics-tls" not found Jun 15 03:09:18.486542 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:18.486395 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 03:09:18.486542 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:18.486492 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6786c7fbc5-8q824: secret "image-registry-tls" not found Jun 15 03:09:18.486542 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:18.486537 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls podName:50f26ba5-125d-4c08-bc95-0fef32a874f6 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:20.486524609 +0000 UTC m=+38.643138818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls") pod "image-registry-6786c7fbc5-8q824" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6") : secret "image-registry-tls" not found Jun 15 03:09:19.477315 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:19.477280 2560 generic.go:356] "Generic (PLEG): container finished" podID="bd7848c5-9bec-441f-bda7-a249bd472e1b" containerID="f60205074665275457d246e9c6152c85438475ecf54c286285af470cd053d2fe" exitCode=0 Jun 15 03:09:19.477741 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:19.477343 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzn4l" event={"ID":"bd7848c5-9bec-441f-bda7-a249bd472e1b","Type":"ContainerDied","Data":"f60205074665275457d246e9c6152c85438475ecf54c286285af470cd053d2fe"} Jun 15 03:09:20.486161 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:20.483475 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzn4l" event={"ID":"bd7848c5-9bec-441f-bda7-a249bd472e1b","Type":"ContainerStarted","Data":"f115b3790147e3287c9fce86e25adcb56493fdb404c30fb6107a0f0982f779bd"} Jun 15 03:09:20.502165 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:20.502136 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:20.502297 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:20.502178 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:20.502369 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:20.502297 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:09:20.502369 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:20.502348 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 03:09:20.502369 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:20.502365 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6786c7fbc5-8q824: secret "image-registry-tls" not found Jun 15 03:09:20.502460 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:20.502392 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls podName:f10bedca-8f8a-4db1-9b58-8d3d854ece5d nodeName:}" failed. No retries permitted until 2026-06-15 03:09:24.502376518 +0000 UTC m=+42.658990704 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls") pod "dns-default-5zmqd" (UID: "f10bedca-8f8a-4db1-9b58-8d3d854ece5d") : secret "dns-default-metrics-tls" not found Jun 15 03:09:20.502460 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:20.502407 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls podName:50f26ba5-125d-4c08-bc95-0fef32a874f6 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:24.502400492 +0000 UTC m=+42.659014677 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls") pod "image-registry-6786c7fbc5-8q824" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6") : secret "image-registry-tls" not found Jun 15 03:09:20.508070 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:20.508022 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rzn4l" podStartSLOduration=8.389144513 podStartE2EDuration="38.508002826s" totalImagesPullingTime="30.118858313s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:20.5078932 +0000 UTC m=+38.664507424" watchObservedRunningTime="2026-06-15 03:09:20.508002826 +0000 UTC m=+38.664617034" Jun 15 03:09:21.492303 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:21.492264 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s7jcv" event={"ID":"5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5","Type":"ContainerStarted","Data":"6a0899d597735d8e0abab831900fc0d22cecf3cb171de4a841d03196fd357540"} Jun 15 03:09:21.508129 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:21.508064 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s7jcv" podStartSLOduration=3.161525243 podStartE2EDuration="5.508049748s" totalImagesPullingTime="2.346524505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:21.50750224 +0000 UTC m=+39.664116449" watchObservedRunningTime="2026-06-15 03:09:21.508049748 +0000 UTC m=+39.664663955" Jun 15 03:09:24.534781 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:24.534601 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:24.535338 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:24.534790 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:24.535338 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:24.534736 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:09:24.535338 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:24.534975 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 03:09:24.535338 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:24.535277 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6786c7fbc5-8q824: secret "image-registry-tls" not found Jun 15 03:09:24.535338 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:24.535295 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls podName:f10bedca-8f8a-4db1-9b58-8d3d854ece5d nodeName:}" failed. No retries permitted until 2026-06-15 03:09:32.535078858 +0000 UTC m=+50.691693043 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls") pod "dns-default-5zmqd" (UID: "f10bedca-8f8a-4db1-9b58-8d3d854ece5d") : secret "dns-default-metrics-tls" not found Jun 15 03:09:24.535607 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:24.535402 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls podName:50f26ba5-125d-4c08-bc95-0fef32a874f6 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:32.535374723 +0000 UTC m=+50.691988922 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls") pod "image-registry-6786c7fbc5-8q824" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6") : secret "image-registry-tls" not found Jun 15 03:09:32.584563 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:32.584530 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:32.584563 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:32.584567 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:32.585042 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:32.584670 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 03:09:32.585042 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:32.584681 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6786c7fbc5-8q824: secret "image-registry-tls" not found Jun 15 03:09:32.585042 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:32.584692 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:09:32.585042 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:32.584733 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls podName:50f26ba5-125d-4c08-bc95-0fef32a874f6 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:48.584721352 +0000 UTC m=+66.741335539 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls") pod "image-registry-6786c7fbc5-8q824" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6") : secret "image-registry-tls" not found Jun 15 03:09:32.585042 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:32.584768 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls podName:f10bedca-8f8a-4db1-9b58-8d3d854ece5d nodeName:}" failed. No retries permitted until 2026-06-15 03:09:48.584749988 +0000 UTC m=+66.741364190 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls") pod "dns-default-5zmqd" (UID: "f10bedca-8f8a-4db1-9b58-8d3d854ece5d") : secret "dns-default-metrics-tls" not found Jun 15 03:09:42.459497 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:42.459470 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vsz59" Jun 15 03:09:47.987569 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:47.987514 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:09:47.988035 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:47.987621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs\") pod \"network-metrics-daemon-9z4f4\" (UID: \"ff14c0e9-ddf7-4720-ac87-54a88354948e\") " pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:09:47.990565 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:47.990544 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 15 03:09:47.990692 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:47.990678 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 15 03:09:47.998701 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:47.998682 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 15 03:09:47.998766 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:47.998754 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs podName:ff14c0e9-ddf7-4720-ac87-54a88354948e nodeName:}" failed. No retries permitted until 2026-06-15 03:10:51.998734403 +0000 UTC m=+130.155348592 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs") pod "network-metrics-daemon-9z4f4" (UID: "ff14c0e9-ddf7-4720-ac87-54a88354948e") : secret "metrics-daemon-secret" not found Jun 15 03:09:48.002402 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:48.002383 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04ea9897-2b15-4a68-93cf-cad10b6c7245-original-pull-secret\") pod \"global-pull-secret-syncer-66qb8\" (UID: \"04ea9897-2b15-4a68-93cf-cad10b6c7245\") " pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:09:48.182287 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:48.182237 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66qb8" Jun 15 03:09:48.189212 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:48.189185 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w55f8\" (UniqueName: \"kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8\") pod \"network-check-target-zm8j4\" (UID: \"249c7536-df2e-4600-b69b-116d1432290a\") " pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:48.193213 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:48.193185 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 15 03:09:48.201756 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:48.201733 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 15 03:09:48.212334 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:48.212312 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w55f8\" (UniqueName: \"kubernetes.io/projected/249c7536-df2e-4600-b69b-116d1432290a-kube-api-access-w55f8\") pod \"network-check-target-zm8j4\" (UID: \"249c7536-df2e-4600-b69b-116d1432290a\") " pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:48.309952 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:48.309922 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-66qb8"] Jun 15 03:09:48.312918 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:09:48.312892 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04ea9897_2b15_4a68_93cf_cad10b6c7245.slice/crio-ebc11ac41904f2b6b428d834ba5c4c430ea915f653f851b81ad073fdcaed502e WatchSource:0}: Error finding container ebc11ac41904f2b6b428d834ba5c4c430ea915f653f851b81ad073fdcaed502e: Status 404 returned error can't find the container with id ebc11ac41904f2b6b428d834ba5c4c430ea915f653f851b81ad073fdcaed502e Jun 15 03:09:48.477498 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:48.477466 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hxgvb\"" Jun 15 03:09:48.485782 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:48.485745 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:48.554709 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:48.554673 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-66qb8" event={"ID":"04ea9897-2b15-4a68-93cf-cad10b6c7245","Type":"ContainerStarted","Data":"ebc11ac41904f2b6b428d834ba5c4c430ea915f653f851b81ad073fdcaed502e"} Jun 15 03:09:48.591769 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:48.591738 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:09:48.591933 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:48.591790 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:09:48.591933 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:48.591842 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:09:48.591933 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:48.591906 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls podName:f10bedca-8f8a-4db1-9b58-8d3d854ece5d nodeName:}" failed. No retries permitted until 2026-06-15 03:10:20.59188866 +0000 UTC m=+98.748502846 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls") pod "dns-default-5zmqd" (UID: "f10bedca-8f8a-4db1-9b58-8d3d854ece5d") : secret "dns-default-metrics-tls" not found Jun 15 03:09:48.592049 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:48.591939 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 03:09:48.592049 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:48.591957 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6786c7fbc5-8q824: secret "image-registry-tls" not found Jun 15 03:09:48.592049 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:09:48.592008 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls podName:50f26ba5-125d-4c08-bc95-0fef32a874f6 nodeName:}" failed. No retries permitted until 2026-06-15 03:10:20.591993311 +0000 UTC m=+98.748607501 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls") pod "image-registry-6786c7fbc5-8q824" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6") : secret "image-registry-tls" not found Jun 15 03:09:48.601377 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:48.601351 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zm8j4"] Jun 15 03:09:48.604205 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:09:48.604183 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod249c7536_df2e_4600_b69b_116d1432290a.slice/crio-6cb8f5cdb42ff729b697c0b031f297d876dbe37bc879f66c1c48ced61507985d WatchSource:0}: Error finding container 6cb8f5cdb42ff729b697c0b031f297d876dbe37bc879f66c1c48ced61507985d: Status 404 returned error can't find the container with id 6cb8f5cdb42ff729b697c0b031f297d876dbe37bc879f66c1c48ced61507985d Jun 15 03:09:49.559147 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:49.559107 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zm8j4" event={"ID":"249c7536-df2e-4600-b69b-116d1432290a","Type":"ContainerStarted","Data":"6cb8f5cdb42ff729b697c0b031f297d876dbe37bc879f66c1c48ced61507985d"} Jun 15 03:09:51.595485 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.595345 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j"] Jun 15 03:09:51.597514 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.597487 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" Jun 15 03:09:51.600186 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.600163 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 15 03:09:51.601248 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.601221 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 15 03:09:51.601248 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.601236 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 15 03:09:51.601443 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.601262 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 15 03:09:51.608305 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.608279 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j"] Jun 15 03:09:51.613003 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.612970 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rs7g\" (UniqueName: \"kubernetes.io/projected/1b0878f7-00b9-4c18-b466-49457c5a4ea9-kube-api-access-4rs7g\") pod \"klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j\" (UID: \"1b0878f7-00b9-4c18-b466-49457c5a4ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" Jun 15 03:09:51.613003 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.613026 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b0878f7-00b9-4c18-b466-49457c5a4ea9-tmp\") pod \"klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j\" (UID: \"1b0878f7-00b9-4c18-b466-49457c5a4ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" Jun 15 03:09:51.613254 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.613099 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1b0878f7-00b9-4c18-b466-49457c5a4ea9-klusterlet-config\") pod \"klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j\" (UID: \"1b0878f7-00b9-4c18-b466-49457c5a4ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" Jun 15 03:09:51.713650 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.713610 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rs7g\" (UniqueName: \"kubernetes.io/projected/1b0878f7-00b9-4c18-b466-49457c5a4ea9-kube-api-access-4rs7g\") pod \"klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j\" (UID: \"1b0878f7-00b9-4c18-b466-49457c5a4ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" Jun 15 03:09:51.713832 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.713667 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b0878f7-00b9-4c18-b466-49457c5a4ea9-tmp\") pod \"klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j\" (UID: \"1b0878f7-00b9-4c18-b466-49457c5a4ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" Jun 15 03:09:51.713832 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.713699 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1b0878f7-00b9-4c18-b466-49457c5a4ea9-klusterlet-config\") pod \"klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j\" (UID: \"1b0878f7-00b9-4c18-b466-49457c5a4ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" Jun 15 03:09:51.714109 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.714076 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b0878f7-00b9-4c18-b466-49457c5a4ea9-tmp\") pod \"klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j\" (UID: \"1b0878f7-00b9-4c18-b466-49457c5a4ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" Jun 15 03:09:51.716513 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.716489 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1b0878f7-00b9-4c18-b466-49457c5a4ea9-klusterlet-config\") pod \"klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j\" (UID: \"1b0878f7-00b9-4c18-b466-49457c5a4ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" Jun 15 03:09:51.722739 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.722715 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rs7g\" (UniqueName: \"kubernetes.io/projected/1b0878f7-00b9-4c18-b466-49457c5a4ea9-kube-api-access-4rs7g\") pod \"klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j\" (UID: \"1b0878f7-00b9-4c18-b466-49457c5a4ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" Jun 15 03:09:51.909670 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:51.909586 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" Jun 15 03:09:53.918814 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:53.918784 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j"] Jun 15 03:09:53.922442 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:09:53.922419 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b0878f7_00b9_4c18_b466_49457c5a4ea9.slice/crio-0335527638a43089c5d4e848ac413a25feadb3472b87c5307eabc11bc09eb413 WatchSource:0}: Error finding container 0335527638a43089c5d4e848ac413a25feadb3472b87c5307eabc11bc09eb413: Status 404 returned error can't find the container with id 0335527638a43089c5d4e848ac413a25feadb3472b87c5307eabc11bc09eb413 Jun 15 03:09:54.573783 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:54.573748 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" event={"ID":"1b0878f7-00b9-4c18-b466-49457c5a4ea9","Type":"ContainerStarted","Data":"0335527638a43089c5d4e848ac413a25feadb3472b87c5307eabc11bc09eb413"} Jun 15 03:09:54.575052 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:54.575028 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zm8j4" event={"ID":"249c7536-df2e-4600-b69b-116d1432290a","Type":"ContainerStarted","Data":"74e0d95fa6fef1117ccf752428705d9fc8decde01792f7b134b7067cb7dffd1c"} Jun 15 03:09:54.575193 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:54.575129 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:09:54.576360 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:54.576340 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-66qb8" event={"ID":"04ea9897-2b15-4a68-93cf-cad10b6c7245","Type":"ContainerStarted","Data":"7d0428011fbdd40db9d192a1640c1d5d24bb1f3cbabe95d884b2359234c65f8f"} Jun 15 03:09:54.591204 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:54.591156 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zm8j4" podStartSLOduration=67.399230018 podStartE2EDuration="1m12.591142306s" totalImagesPullingTime="5.191912288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:54.590701249 +0000 UTC m=+72.747315470" watchObservedRunningTime="2026-06-15 03:09:54.591142306 +0000 UTC m=+72.747756551" Jun 15 03:09:54.604230 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:54.604190 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-66qb8" podStartSLOduration=67.120334504 podStartE2EDuration="1m12.604177998s" totalImagesPullingTime="5.483843494s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:54.604016635 +0000 UTC m=+72.760630843" watchObservedRunningTime="2026-06-15 03:09:54.604177998 +0000 UTC m=+72.760792207" Jun 15 03:09:58.588270 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:58.588219 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" event={"ID":"1b0878f7-00b9-4c18-b466-49457c5a4ea9","Type":"ContainerStarted","Data":"4e78171293804f47c7a89121fbf051cef9365ebb7244952562ebb3ab616d2b22"} Jun 15 03:09:58.588705 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:58.588450 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" Jun 15 03:09:58.590070 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:58.590049 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" Jun 15 03:09:58.605231 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:09:58.605186 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" podStartSLOduration=3.420943233 podStartE2EDuration="7.605168911s" totalImagesPullingTime="4.184225678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:58.60358594 +0000 UTC m=+76.760200149" watchObservedRunningTime="2026-06-15 03:09:58.605168911 +0000 UTC m=+76.761783119" Jun 15 03:10:20.614759 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:10:20.614720 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:10:20.614759 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:10:20.614761 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:10:20.615309 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:10:20.614867 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:10:20.615309 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:10:20.614881 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 03:10:20.615309 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:10:20.614894 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6786c7fbc5-8q824: secret "image-registry-tls" not found Jun 15 03:10:20.615309 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:10:20.614973 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls podName:f10bedca-8f8a-4db1-9b58-8d3d854ece5d nodeName:}" failed. No retries permitted until 2026-06-15 03:11:24.614956765 +0000 UTC m=+162.771570951 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls") pod "dns-default-5zmqd" (UID: "f10bedca-8f8a-4db1-9b58-8d3d854ece5d") : secret "dns-default-metrics-tls" not found Jun 15 03:10:20.615309 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:10:20.615046 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls podName:50f26ba5-125d-4c08-bc95-0fef32a874f6 nodeName:}" failed. No retries permitted until 2026-06-15 03:11:24.615031129 +0000 UTC m=+162.771645322 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls") pod "image-registry-6786c7fbc5-8q824" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6") : secret "image-registry-tls" not found Jun 15 03:10:25.582934 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:10:25.582895 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zm8j4" Jun 15 03:10:52.026647 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:10:52.026587 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs\") pod \"network-metrics-daemon-9z4f4\" (UID: \"ff14c0e9-ddf7-4720-ac87-54a88354948e\") " pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:10:52.027155 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:10:52.026738 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 15 03:10:52.027155 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:10:52.026811 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs podName:ff14c0e9-ddf7-4720-ac87-54a88354948e nodeName:}" failed. No retries permitted until 2026-06-15 03:12:54.026796982 +0000 UTC m=+252.183411168 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs") pod "network-metrics-daemon-9z4f4" (UID: "ff14c0e9-ddf7-4720-ac87-54a88354948e") : secret "metrics-daemon-secret" not found Jun 15 03:11:19.748203 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:11:19.748147 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" podUID="50f26ba5-125d-4c08-bc95-0fef32a874f6" Jun 15 03:11:19.764284 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:11:19.764256 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5zmqd" podUID="f10bedca-8f8a-4db1-9b58-8d3d854ece5d" Jun 15 03:11:19.775019 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:11:19.774996 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zmqd" Jun 15 03:11:19.775142 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:11:19.774996 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:11:20.286891 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:11:20.286857 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9z4f4" podUID="ff14c0e9-ddf7-4720-ac87-54a88354948e" Jun 15 03:11:24.660303 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:11:24.660223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:11:24.660303 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:11:24.660261 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:11:24.660753 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:11:24.660373 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:11:24.660753 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:11:24.660427 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls podName:f10bedca-8f8a-4db1-9b58-8d3d854ece5d nodeName:}" failed. No retries permitted until 2026-06-15 03:13:26.660414496 +0000 UTC m=+284.817028695 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls") pod "dns-default-5zmqd" (UID: "f10bedca-8f8a-4db1-9b58-8d3d854ece5d") : secret "dns-default-metrics-tls" not found Jun 15 03:11:24.660753 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:11:24.660377 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 03:11:24.660753 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:11:24.660449 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6786c7fbc5-8q824: secret "image-registry-tls" not found Jun 15 03:11:24.660753 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:11:24.660482 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls podName:50f26ba5-125d-4c08-bc95-0fef32a874f6 nodeName:}" failed. No retries permitted until 2026-06-15 03:13:26.660473378 +0000 UTC m=+284.817087577 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls") pod "image-registry-6786c7fbc5-8q824" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6") : secret "image-registry-tls" not found Jun 15 03:11:33.261431 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:11:33.261379 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:11:58.589067 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:11:58.589020 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" podUID="1b0878f7-00b9-4c18-b466-49457c5a4ea9" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.9:8000/readyz\": dial tcp 10.132.0.9:8000: connect: connection refused" Jun 15 03:11:58.866534 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:11:58.866455 2560 generic.go:356] "Generic (PLEG): container finished" podID="1b0878f7-00b9-4c18-b466-49457c5a4ea9" containerID="4e78171293804f47c7a89121fbf051cef9365ebb7244952562ebb3ab616d2b22" exitCode=1 Jun 15 03:11:58.866534 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:11:58.866514 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" event={"ID":"1b0878f7-00b9-4c18-b466-49457c5a4ea9","Type":"ContainerDied","Data":"4e78171293804f47c7a89121fbf051cef9365ebb7244952562ebb3ab616d2b22"} Jun 15 03:11:58.866802 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:11:58.866790 2560 scope.go:117] "RemoveContainer" containerID="4e78171293804f47c7a89121fbf051cef9365ebb7244952562ebb3ab616d2b22" Jun 15 03:11:59.870758 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:11:59.870720 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" event={"ID":"1b0878f7-00b9-4c18-b466-49457c5a4ea9","Type":"ContainerStarted","Data":"1156706ed5199c62774ad7153ae2ca08400d872e2b2bc782f669990a246c3fe1"} Jun 15 03:11:59.871273 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:11:59.870986 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" Jun 15 03:11:59.871646 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:11:59.871628 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bc6bc4bdd-9fc8j" Jun 15 03:12:01.148629 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.148595 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb"] Jun 15 03:12:01.150411 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.150394 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:01.153259 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.153235 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 15 03:12:01.157427 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.155899 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-blh42\"" Jun 15 03:12:01.157427 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.156255 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 15 03:12:01.157587 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.157478 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 15 03:12:01.157587 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.157526 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 15 03:12:01.159278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.159245 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-t6php"] Jun 15 03:12:01.161121 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.161103 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-t6php" Jun 15 03:12:01.163545 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.163526 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 15 03:12:01.163624 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.163600 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-f7h9g\"" Jun 15 03:12:01.163921 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.163907 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 15 03:12:01.167973 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.167952 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb"] Jun 15 03:12:01.174968 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.174947 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-t6php"] Jun 15 03:12:01.210328 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.210301 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-9mvkb\" (UID: \"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:01.210485 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.210351 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-telemetry-config\") pod \"cluster-monitoring-operator-f9d7df769-9mvkb\" (UID: \"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:01.210485 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.210375 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sfsm\" (UniqueName: \"kubernetes.io/projected/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-kube-api-access-7sfsm\") pod \"cluster-monitoring-operator-f9d7df769-9mvkb\" (UID: \"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:01.210485 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.210393 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-224nl\" (UniqueName: \"kubernetes.io/projected/4ae9d1ef-8309-47e6-a27d-e57f590e0b2b-kube-api-access-224nl\") pod \"volume-data-source-validator-bd4ccd799-t6php\" (UID: \"4ae9d1ef-8309-47e6-a27d-e57f590e0b2b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-t6php" Jun 15 03:12:01.250632 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.250603 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-55f78b5b6d-xfjzj"] Jun 15 03:12:01.252478 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.252463 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.255246 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.255227 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 15 03:12:01.256170 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.256150 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6ksmh\"" Jun 15 03:12:01.256415 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.256386 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 15 03:12:01.256507 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.256476 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 15 03:12:01.256507 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.256497 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 15 03:12:01.257204 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.257183 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 15 03:12:01.257302 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.257219 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 15 03:12:01.269579 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.269559 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-55f78b5b6d-xfjzj"] Jun 15 03:12:01.311504 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.311472 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.311676 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.311510 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-stats-auth\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.311676 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.311610 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-9mvkb\" (UID: \"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:01.311676 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.311637 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-default-certificate\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.311676 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.311675 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-telemetry-config\") pod \"cluster-monitoring-operator-f9d7df769-9mvkb\" (UID: \"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:01.311875 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.311711 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sfsm\" (UniqueName: \"kubernetes.io/projected/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-kube-api-access-7sfsm\") pod \"cluster-monitoring-operator-f9d7df769-9mvkb\" (UID: \"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:01.311875 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.311729 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.311875 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.311745 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6ngh\" (UniqueName: \"kubernetes.io/projected/1429058a-5113-4842-83d5-1f57f1fee24a-kube-api-access-r6ngh\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.311875 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:01.311754 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 15 03:12:01.311875 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.311763 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-224nl\" (UniqueName: \"kubernetes.io/projected/4ae9d1ef-8309-47e6-a27d-e57f590e0b2b-kube-api-access-224nl\") pod \"volume-data-source-validator-bd4ccd799-t6php\" (UID: \"4ae9d1ef-8309-47e6-a27d-e57f590e0b2b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-t6php" Jun 15 03:12:01.311875 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:01.311828 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls podName:cde73dc3-0cfa-4b1b-afe1-296bdf4e026c nodeName:}" failed. No retries permitted until 2026-06-15 03:12:01.811808581 +0000 UTC m=+199.968422781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-9mvkb" (UID: "cde73dc3-0cfa-4b1b-afe1-296bdf4e026c") : secret "cluster-monitoring-operator-tls" not found Jun 15 03:12:01.312422 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.312400 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-telemetry-config\") pod \"cluster-monitoring-operator-f9d7df769-9mvkb\" (UID: \"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:01.319994 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.319974 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sfsm\" (UniqueName: \"kubernetes.io/projected/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-kube-api-access-7sfsm\") pod \"cluster-monitoring-operator-f9d7df769-9mvkb\" (UID: \"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:01.320143 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.320128 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-224nl\" (UniqueName: \"kubernetes.io/projected/4ae9d1ef-8309-47e6-a27d-e57f590e0b2b-kube-api-access-224nl\") pod \"volume-data-source-validator-bd4ccd799-t6php\" (UID: \"4ae9d1ef-8309-47e6-a27d-e57f590e0b2b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-t6php" Jun 15 03:12:01.344442 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.344422 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-867b76cb74-n9b2r"] Jun 15 03:12:01.346219 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.346206 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" Jun 15 03:12:01.348679 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.348663 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 15 03:12:01.348910 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.348895 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qsv6k\"" Jun 15 03:12:01.349041 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.349026 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 15 03:12:01.357030 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.357003 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-867b76cb74-n9b2r"] Jun 15 03:12:01.412050 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.411988 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cf2910da-c1b3-4520-92be-4e1b96f6d878-nginx-conf\") pod \"networking-console-plugin-867b76cb74-n9b2r\" (UID: \"cf2910da-c1b3-4520-92be-4e1b96f6d878\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" Jun 15 03:12:01.412050 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.412022 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-n9b2r\" (UID: \"cf2910da-c1b3-4520-92be-4e1b96f6d878\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" Jun 15 03:12:01.412221 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.412052 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.412221 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.412115 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6ngh\" (UniqueName: \"kubernetes.io/projected/1429058a-5113-4842-83d5-1f57f1fee24a-kube-api-access-r6ngh\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.412221 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.412144 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.412221 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:01.412187 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle podName:1429058a-5113-4842-83d5-1f57f1fee24a nodeName:}" failed. No retries permitted until 2026-06-15 03:12:01.912169383 +0000 UTC m=+200.068783584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle") pod "router-default-55f78b5b6d-xfjzj" (UID: "1429058a-5113-4842-83d5-1f57f1fee24a") : configmap references non-existent config key: service-ca.crt Jun 15 03:12:01.412221 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.412215 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-default-certificate\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.412468 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:01.412233 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 15 03:12:01.412468 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:01.412271 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs podName:1429058a-5113-4842-83d5-1f57f1fee24a nodeName:}" failed. No retries permitted until 2026-06-15 03:12:01.912258061 +0000 UTC m=+200.068872265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs") pod "router-default-55f78b5b6d-xfjzj" (UID: "1429058a-5113-4842-83d5-1f57f1fee24a") : secret "router-metrics-certs-default" not found Jun 15 03:12:01.412468 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.412318 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-stats-auth\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.414670 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.414647 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-stats-auth\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.414767 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.414651 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-default-certificate\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.420320 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.420298 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6ngh\" (UniqueName: \"kubernetes.io/projected/1429058a-5113-4842-83d5-1f57f1fee24a-kube-api-access-r6ngh\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.469584 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.469556 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-t6php" Jun 15 03:12:01.513348 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.513309 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cf2910da-c1b3-4520-92be-4e1b96f6d878-nginx-conf\") pod \"networking-console-plugin-867b76cb74-n9b2r\" (UID: \"cf2910da-c1b3-4520-92be-4e1b96f6d878\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" Jun 15 03:12:01.513482 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.513363 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-n9b2r\" (UID: \"cf2910da-c1b3-4520-92be-4e1b96f6d878\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" Jun 15 03:12:01.513575 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:01.513556 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 15 03:12:01.513651 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:01.513636 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert podName:cf2910da-c1b3-4520-92be-4e1b96f6d878 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:02.013616886 +0000 UTC m=+200.170231092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-n9b2r" (UID: "cf2910da-c1b3-4520-92be-4e1b96f6d878") : secret "networking-console-plugin-cert" not found Jun 15 03:12:01.514192 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.514164 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cf2910da-c1b3-4520-92be-4e1b96f6d878-nginx-conf\") pod \"networking-console-plugin-867b76cb74-n9b2r\" (UID: \"cf2910da-c1b3-4520-92be-4e1b96f6d878\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" Jun 15 03:12:01.585818 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.585778 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-t6php"] Jun 15 03:12:01.589283 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:12:01.589258 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ae9d1ef_8309_47e6_a27d_e57f590e0b2b.slice/crio-5e861671def09a62e1f5abdaa90840f123b29ab9d2baf3de399db7def8b2a2de WatchSource:0}: Error finding container 5e861671def09a62e1f5abdaa90840f123b29ab9d2baf3de399db7def8b2a2de: Status 404 returned error can't find the container with id 5e861671def09a62e1f5abdaa90840f123b29ab9d2baf3de399db7def8b2a2de Jun 15 03:12:01.815733 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.815690 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-9mvkb\" (UID: \"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:01.815892 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:01.815836 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 15 03:12:01.815930 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:01.815898 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls podName:cde73dc3-0cfa-4b1b-afe1-296bdf4e026c nodeName:}" failed. No retries permitted until 2026-06-15 03:12:02.815883452 +0000 UTC m=+200.972497641 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-9mvkb" (UID: "cde73dc3-0cfa-4b1b-afe1-296bdf4e026c") : secret "cluster-monitoring-operator-tls" not found Jun 15 03:12:01.878255 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.878216 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-t6php" event={"ID":"4ae9d1ef-8309-47e6-a27d-e57f590e0b2b","Type":"ContainerStarted","Data":"5e861671def09a62e1f5abdaa90840f123b29ab9d2baf3de399db7def8b2a2de"} Jun 15 03:12:01.916754 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.916709 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.916754 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:01.916760 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:01.916979 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:01.916833 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle podName:1429058a-5113-4842-83d5-1f57f1fee24a nodeName:}" failed. No retries permitted until 2026-06-15 03:12:02.916812072 +0000 UTC m=+201.073426262 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle") pod "router-default-55f78b5b6d-xfjzj" (UID: "1429058a-5113-4842-83d5-1f57f1fee24a") : configmap references non-existent config key: service-ca.crt Jun 15 03:12:01.916979 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:01.916881 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 15 03:12:01.916979 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:01.916931 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs podName:1429058a-5113-4842-83d5-1f57f1fee24a nodeName:}" failed. No retries permitted until 2026-06-15 03:12:02.91691807 +0000 UTC m=+201.073532256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs") pod "router-default-55f78b5b6d-xfjzj" (UID: "1429058a-5113-4842-83d5-1f57f1fee24a") : secret "router-metrics-certs-default" not found Jun 15 03:12:02.017943 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:02.017897 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-n9b2r\" (UID: \"cf2910da-c1b3-4520-92be-4e1b96f6d878\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" Jun 15 03:12:02.018152 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:02.018040 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 15 03:12:02.018152 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:02.018140 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert podName:cf2910da-c1b3-4520-92be-4e1b96f6d878 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:03.018115743 +0000 UTC m=+201.174729955 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-n9b2r" (UID: "cf2910da-c1b3-4520-92be-4e1b96f6d878") : secret "networking-console-plugin-cert" not found Jun 15 03:12:02.824841 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:02.824799 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-9mvkb\" (UID: \"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:02.825354 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:02.824972 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 15 03:12:02.825354 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:02.825059 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls podName:cde73dc3-0cfa-4b1b-afe1-296bdf4e026c nodeName:}" failed. No retries permitted until 2026-06-15 03:12:04.825039567 +0000 UTC m=+202.981653770 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-9mvkb" (UID: "cde73dc3-0cfa-4b1b-afe1-296bdf4e026c") : secret "cluster-monitoring-operator-tls" not found Jun 15 03:12:02.882326 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:02.882291 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-t6php" event={"ID":"4ae9d1ef-8309-47e6-a27d-e57f590e0b2b","Type":"ContainerStarted","Data":"e2ecd7a84130e619b028554d26b5373d6068434617cc8ec8222e8164fa9f6f36"} Jun 15 03:12:02.897911 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:02.897859 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-t6php" podStartSLOduration=1.064880591 podStartE2EDuration="1.89784444s" totalImagesPullingTime="832.963849ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:02.897014125 +0000 UTC m=+201.053628333" watchObservedRunningTime="2026-06-15 03:12:02.89784444 +0000 UTC m=+201.054458647" Jun 15 03:12:02.925335 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:02.925303 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:02.925488 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:02.925347 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:02.925488 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:02.925459 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 15 03:12:02.925590 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:02.925461 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle podName:1429058a-5113-4842-83d5-1f57f1fee24a nodeName:}" failed. No retries permitted until 2026-06-15 03:12:04.925446364 +0000 UTC m=+203.082060554 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle") pod "router-default-55f78b5b6d-xfjzj" (UID: "1429058a-5113-4842-83d5-1f57f1fee24a") : configmap references non-existent config key: service-ca.crt Jun 15 03:12:02.925590 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:02.925520 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs podName:1429058a-5113-4842-83d5-1f57f1fee24a nodeName:}" failed. No retries permitted until 2026-06-15 03:12:04.925500045 +0000 UTC m=+203.082114240 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs") pod "router-default-55f78b5b6d-xfjzj" (UID: "1429058a-5113-4842-83d5-1f57f1fee24a") : secret "router-metrics-certs-default" not found Jun 15 03:12:03.026441 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:03.026392 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-n9b2r\" (UID: \"cf2910da-c1b3-4520-92be-4e1b96f6d878\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" Jun 15 03:12:03.026637 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:03.026525 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 15 03:12:03.026637 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:03.026595 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert podName:cf2910da-c1b3-4520-92be-4e1b96f6d878 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:05.026575211 +0000 UTC m=+203.183189399 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-n9b2r" (UID: "cf2910da-c1b3-4520-92be-4e1b96f6d878") : secret "networking-console-plugin-cert" not found Jun 15 03:12:04.839616 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:04.839553 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-9mvkb\" (UID: \"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:04.839983 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:04.839719 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 15 03:12:04.839983 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:04.839788 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls podName:cde73dc3-0cfa-4b1b-afe1-296bdf4e026c nodeName:}" failed. No retries permitted until 2026-06-15 03:12:08.839773245 +0000 UTC m=+206.996387431 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-9mvkb" (UID: "cde73dc3-0cfa-4b1b-afe1-296bdf4e026c") : secret "cluster-monitoring-operator-tls" not found Jun 15 03:12:04.940725 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:04.940672 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:04.940725 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:04.940732 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:04.940930 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:04.940817 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 15 03:12:04.940930 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:04.940866 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs podName:1429058a-5113-4842-83d5-1f57f1fee24a nodeName:}" failed. No retries permitted until 2026-06-15 03:12:08.940852047 +0000 UTC m=+207.097466234 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs") pod "router-default-55f78b5b6d-xfjzj" (UID: "1429058a-5113-4842-83d5-1f57f1fee24a") : secret "router-metrics-certs-default" not found Jun 15 03:12:04.940930 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:04.940885 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle podName:1429058a-5113-4842-83d5-1f57f1fee24a nodeName:}" failed. No retries permitted until 2026-06-15 03:12:08.940878976 +0000 UTC m=+207.097493161 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle") pod "router-default-55f78b5b6d-xfjzj" (UID: "1429058a-5113-4842-83d5-1f57f1fee24a") : configmap references non-existent config key: service-ca.crt Jun 15 03:12:05.042055 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.042002 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-n9b2r\" (UID: \"cf2910da-c1b3-4520-92be-4e1b96f6d878\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" Jun 15 03:12:05.042186 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:05.042164 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 15 03:12:05.042239 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:05.042231 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert podName:cf2910da-c1b3-4520-92be-4e1b96f6d878 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:09.042216226 +0000 UTC m=+207.198830417 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-n9b2r" (UID: "cf2910da-c1b3-4520-92be-4e1b96f6d878") : secret "networking-console-plugin-cert" not found Jun 15 03:12:05.612038 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.612010 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qmv4z"] Jun 15 03:12:05.613967 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.613953 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:05.616647 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.616624 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 15 03:12:05.616862 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.616847 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 15 03:12:05.617216 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.617173 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 15 03:12:05.617388 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.617376 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 15 03:12:05.618069 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.618051 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nvxls\"" Jun 15 03:12:05.631039 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.631013 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qmv4z"] Jun 15 03:12:05.748387 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.748351 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-data-volume\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:05.748569 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.748420 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-crio-socket\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:05.748569 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.748442 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jfr7\" (UniqueName: \"kubernetes.io/projected/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-api-access-5jfr7\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:05.748569 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.748466 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:05.748569 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.748558 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:05.849844 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.849796 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-crio-socket\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:05.849844 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.849847 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jfr7\" (UniqueName: \"kubernetes.io/projected/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-api-access-5jfr7\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:05.850329 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.849920 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-crio-socket\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:05.850329 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.849975 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:05.850329 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:05.850073 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 03:12:05.850329 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.850101 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:05.850329 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:05.850151 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm podName:2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:06.350135238 +0000 UTC m=+204.506749424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qmv4z" (UID: "2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307") : configmap "kube-rbac-proxy" not found Jun 15 03:12:05.850329 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.850201 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-data-volume\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:05.850329 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:05.850213 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 03:12:05.850329 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:05.850261 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls podName:2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:06.350245583 +0000 UTC m=+204.506859774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmv4z" (UID: "2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307") : secret "insights-runtime-extractor-tls" not found Jun 15 03:12:05.850580 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.850454 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-data-volume\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:05.859018 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:05.858995 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jfr7\" (UniqueName: \"kubernetes.io/projected/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-api-access-5jfr7\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:06.354721 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:06.354693 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:06.354889 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:06.354758 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:06.354889 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:06.354800 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 03:12:06.354889 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:06.354852 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 03:12:06.354889 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:06.354873 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm podName:2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:07.354857893 +0000 UTC m=+205.511472082 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qmv4z" (UID: "2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307") : configmap "kube-rbac-proxy" not found Jun 15 03:12:06.355028 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:06.354896 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls podName:2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:07.354885871 +0000 UTC m=+205.511500057 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmv4z" (UID: "2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307") : secret "insights-runtime-extractor-tls" not found Jun 15 03:12:07.362333 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:07.362285 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:07.362704 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:07.362371 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:07.362704 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:07.362409 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 03:12:07.362704 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:07.362473 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 03:12:07.362704 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:07.362489 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm podName:2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:09.362473419 +0000 UTC m=+207.519087605 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qmv4z" (UID: "2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307") : configmap "kube-rbac-proxy" not found Jun 15 03:12:07.362704 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:07.362511 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls podName:2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:09.362499851 +0000 UTC m=+207.519114038 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmv4z" (UID: "2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307") : secret "insights-runtime-extractor-tls" not found Jun 15 03:12:07.902227 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:07.902200 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zq7r8_48bd8c21-f7a6-47fb-adda-1f4d03b37c64/dns-node-resolver/0.log" Jun 15 03:12:08.702546 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:08.702518 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vzjh4_5c55e28b-e913-482a-b2ef-e9bf7d2e78e8/node-ca/0.log" Jun 15 03:12:08.873244 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:08.873209 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-9mvkb\" (UID: \"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:08.873378 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:08.873359 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 15 03:12:08.873432 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:08.873425 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls podName:cde73dc3-0cfa-4b1b-afe1-296bdf4e026c nodeName:}" failed. No retries permitted until 2026-06-15 03:12:16.873409267 +0000 UTC m=+215.030023453 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-9mvkb" (UID: "cde73dc3-0cfa-4b1b-afe1-296bdf4e026c") : secret "cluster-monitoring-operator-tls" not found Jun 15 03:12:08.974169 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:08.974095 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:08.974169 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:08.974131 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:08.974329 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:08.974249 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle podName:1429058a-5113-4842-83d5-1f57f1fee24a nodeName:}" failed. No retries permitted until 2026-06-15 03:12:16.974232582 +0000 UTC m=+215.130846768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle") pod "router-default-55f78b5b6d-xfjzj" (UID: "1429058a-5113-4842-83d5-1f57f1fee24a") : configmap references non-existent config key: service-ca.crt Jun 15 03:12:08.974329 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:08.974293 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 15 03:12:08.974400 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:08.974337 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs podName:1429058a-5113-4842-83d5-1f57f1fee24a nodeName:}" failed. No retries permitted until 2026-06-15 03:12:16.974325291 +0000 UTC m=+215.130939483 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs") pod "router-default-55f78b5b6d-xfjzj" (UID: "1429058a-5113-4842-83d5-1f57f1fee24a") : secret "router-metrics-certs-default" not found Jun 15 03:12:09.074904 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:09.074868 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-n9b2r\" (UID: \"cf2910da-c1b3-4520-92be-4e1b96f6d878\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" Jun 15 03:12:09.075017 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:09.075003 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 15 03:12:09.075076 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:09.075066 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert podName:cf2910da-c1b3-4520-92be-4e1b96f6d878 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:17.075053069 +0000 UTC m=+215.231667255 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-n9b2r" (UID: "cf2910da-c1b3-4520-92be-4e1b96f6d878") : secret "networking-console-plugin-cert" not found Jun 15 03:12:09.378076 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:09.378043 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:09.378278 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:09.378132 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:09.378278 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:09.378176 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 03:12:09.378278 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:09.378251 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm podName:2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:13.378233058 +0000 UTC m=+211.534847258 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qmv4z" (UID: "2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307") : configmap "kube-rbac-proxy" not found Jun 15 03:12:09.378278 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:09.378269 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 03:12:09.378506 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:09.378341 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls podName:2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:13.378326131 +0000 UTC m=+211.534940337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmv4z" (UID: "2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307") : secret "insights-runtime-extractor-tls" not found Jun 15 03:12:09.703408 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:09.703307 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s7jcv_5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5/serve-healthcheck-canary/0.log" Jun 15 03:12:13.409197 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:13.409148 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:13.409554 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:13.409228 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:13.409554 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:13.409275 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 03:12:13.409554 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:13.409324 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 03:12:13.409554 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:13.409348 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm podName:2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:21.409330971 +0000 UTC m=+219.565945166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qmv4z" (UID: "2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307") : configmap "kube-rbac-proxy" not found Jun 15 03:12:13.409554 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:13.409363 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls podName:2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:21.409357654 +0000 UTC m=+219.565971840 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmv4z" (UID: "2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307") : secret "insights-runtime-extractor-tls" not found Jun 15 03:12:16.937048 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:16.937017 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-9mvkb\" (UID: \"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:16.937470 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:16.937176 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 15 03:12:16.937470 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:16.937234 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls podName:cde73dc3-0cfa-4b1b-afe1-296bdf4e026c nodeName:}" failed. No retries permitted until 2026-06-15 03:12:32.937219185 +0000 UTC m=+231.093833371 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-9mvkb" (UID: "cde73dc3-0cfa-4b1b-afe1-296bdf4e026c") : secret "cluster-monitoring-operator-tls" not found Jun 15 03:12:17.037708 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:17.037676 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:17.037852 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:17.037837 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:17.037966 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:17.037953 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle podName:1429058a-5113-4842-83d5-1f57f1fee24a nodeName:}" failed. No retries permitted until 2026-06-15 03:12:33.03793411 +0000 UTC m=+231.194548302 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle") pod "router-default-55f78b5b6d-xfjzj" (UID: "1429058a-5113-4842-83d5-1f57f1fee24a") : configmap references non-existent config key: service-ca.crt Jun 15 03:12:17.040161 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:17.040129 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1429058a-5113-4842-83d5-1f57f1fee24a-metrics-certs\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:17.138639 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:17.138602 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-n9b2r\" (UID: \"cf2910da-c1b3-4520-92be-4e1b96f6d878\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" Jun 15 03:12:17.138804 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:17.138757 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 15 03:12:17.138846 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:17.138820 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert podName:cf2910da-c1b3-4520-92be-4e1b96f6d878 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:33.138805046 +0000 UTC m=+231.295419236 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-n9b2r" (UID: "cf2910da-c1b3-4520-92be-4e1b96f6d878") : secret "networking-console-plugin-cert" not found Jun 15 03:12:21.472836 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:21.472800 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:21.473234 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:21.472912 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:21.473234 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:21.472954 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 03:12:21.473234 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:21.473026 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls podName:2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:37.47300547 +0000 UTC m=+235.629619660 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmv4z" (UID: "2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307") : secret "insights-runtime-extractor-tls" not found Jun 15 03:12:21.473500 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:21.473482 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:32.960528 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:32.960479 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-9mvkb\" (UID: \"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:32.962850 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:32.962828 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cde73dc3-0cfa-4b1b-afe1-296bdf4e026c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-9mvkb\" (UID: \"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:33.061909 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:33.061877 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:33.062433 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:33.062416 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1429058a-5113-4842-83d5-1f57f1fee24a-service-ca-bundle\") pod \"router-default-55f78b5b6d-xfjzj\" (UID: \"1429058a-5113-4842-83d5-1f57f1fee24a\") " pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:33.162925 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:33.162893 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-n9b2r\" (UID: \"cf2910da-c1b3-4520-92be-4e1b96f6d878\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" Jun 15 03:12:33.165169 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:33.165139 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf2910da-c1b3-4520-92be-4e1b96f6d878-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-n9b2r\" (UID: \"cf2910da-c1b3-4520-92be-4e1b96f6d878\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" Jun 15 03:12:33.262416 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:33.262340 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" Jun 15 03:12:33.361450 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:33.361422 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:33.382577 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:33.382546 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb"] Jun 15 03:12:33.383813 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:12:33.383783 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcde73dc3_0cfa_4b1b_afe1_296bdf4e026c.slice/crio-2d60bfb1308e965d630024ff7647d8c9216a1040889427d94606073b0bb9c822 WatchSource:0}: Error finding container 2d60bfb1308e965d630024ff7647d8c9216a1040889427d94606073b0bb9c822: Status 404 returned error can't find the container with id 2d60bfb1308e965d630024ff7647d8c9216a1040889427d94606073b0bb9c822 Jun 15 03:12:33.454356 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:33.454328 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" Jun 15 03:12:33.483968 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:33.483943 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-55f78b5b6d-xfjzj"] Jun 15 03:12:33.486065 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:12:33.486040 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1429058a_5113_4842_83d5_1f57f1fee24a.slice/crio-8bbacebc345969fce33ad80daa45e5367fa03f3609ad2a4f42fb47739fcbd3a6 WatchSource:0}: Error finding container 8bbacebc345969fce33ad80daa45e5367fa03f3609ad2a4f42fb47739fcbd3a6: Status 404 returned error can't find the container with id 8bbacebc345969fce33ad80daa45e5367fa03f3609ad2a4f42fb47739fcbd3a6 Jun 15 03:12:33.574239 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:33.574211 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-867b76cb74-n9b2r"] Jun 15 03:12:33.576893 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:12:33.576862 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf2910da_c1b3_4520_92be_4e1b96f6d878.slice/crio-cb0ec807d0d8f30652e9e8c00ac78363e124f1952244cf19957169821157103f WatchSource:0}: Error finding container cb0ec807d0d8f30652e9e8c00ac78363e124f1952244cf19957169821157103f: Status 404 returned error can't find the container with id cb0ec807d0d8f30652e9e8c00ac78363e124f1952244cf19957169821157103f Jun 15 03:12:33.966842 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:33.966804 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" event={"ID":"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c","Type":"ContainerStarted","Data":"2d60bfb1308e965d630024ff7647d8c9216a1040889427d94606073b0bb9c822"} Jun 15 03:12:33.967847 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:33.967817 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" event={"ID":"cf2910da-c1b3-4520-92be-4e1b96f6d878","Type":"ContainerStarted","Data":"cb0ec807d0d8f30652e9e8c00ac78363e124f1952244cf19957169821157103f"} Jun 15 03:12:33.969263 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:33.969234 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" event={"ID":"1429058a-5113-4842-83d5-1f57f1fee24a","Type":"ContainerStarted","Data":"d6b25afaefbfbc1246936ec5a0d533b4bfacd7ad644e8649850805eba8f6a8c1"} Jun 15 03:12:33.969349 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:33.969276 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" event={"ID":"1429058a-5113-4842-83d5-1f57f1fee24a","Type":"ContainerStarted","Data":"8bbacebc345969fce33ad80daa45e5367fa03f3609ad2a4f42fb47739fcbd3a6"} Jun 15 03:12:33.989743 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:33.989698 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" podStartSLOduration=32.989683532 podStartE2EDuration="32.989683532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:33.988654762 +0000 UTC m=+232.145268970" watchObservedRunningTime="2026-06-15 03:12:33.989683532 +0000 UTC m=+232.146297739" Jun 15 03:12:34.361843 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:34.361808 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:34.364569 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:34.364546 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:34.972877 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:34.972797 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:34.974226 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:34.974203 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-55f78b5b6d-xfjzj" Jun 15 03:12:35.667484 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:35.667447 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7hkl2"] Jun 15 03:12:35.670055 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:35.670033 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7hkl2" Jun 15 03:12:35.672793 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:35.672768 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-xmgxf\"" Jun 15 03:12:35.672890 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:35.672807 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 15 03:12:35.681632 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:35.681608 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7hkl2"] Jun 15 03:12:35.681999 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:35.681979 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/35718fb0-c60a-4206-a17f-746409b06664-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-7hkl2\" (UID: \"35718fb0-c60a-4206-a17f-746409b06664\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7hkl2" Jun 15 03:12:35.782431 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:35.782400 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/35718fb0-c60a-4206-a17f-746409b06664-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-7hkl2\" (UID: \"35718fb0-c60a-4206-a17f-746409b06664\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7hkl2" Jun 15 03:12:35.782597 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:35.782543 2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jun 15 03:12:35.782658 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:35.782609 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35718fb0-c60a-4206-a17f-746409b06664-tls-certificates podName:35718fb0-c60a-4206-a17f-746409b06664 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:36.282587581 +0000 UTC m=+234.439201772 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/35718fb0-c60a-4206-a17f-746409b06664-tls-certificates") pod "prometheus-operator-admission-webhook-5cf667749c-7hkl2" (UID: "35718fb0-c60a-4206-a17f-746409b06664") : secret "prometheus-operator-admission-webhook-tls" not found Jun 15 03:12:35.977296 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:35.977209 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" event={"ID":"cde73dc3-0cfa-4b1b-afe1-296bdf4e026c","Type":"ContainerStarted","Data":"f692e67f7774bfffc68ee7626c1213f1a2743dc10ac1c729ffac02d746076117"} Jun 15 03:12:35.978579 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:35.978553 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" event={"ID":"cf2910da-c1b3-4520-92be-4e1b96f6d878","Type":"ContainerStarted","Data":"1f494563d63af3ee8dd32ab8cfec46b6b0225fdef1a5b36d842ff4c1af7f41c8"} Jun 15 03:12:35.994135 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:35.994075 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-9mvkb" podStartSLOduration=33.174534616 podStartE2EDuration="34.994062475s" totalImagesPullingTime="1.819527859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:35.992646907 +0000 UTC m=+234.149261116" watchObservedRunningTime="2026-06-15 03:12:35.994062475 +0000 UTC m=+234.150676683" Jun 15 03:12:36.007180 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:36.007140 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-867b76cb74-n9b2r" podStartSLOduration=33.87906229 podStartE2EDuration="35.007126755s" totalImagesPullingTime="1.128064465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:36.006385071 +0000 UTC m=+234.162999279" watchObservedRunningTime="2026-06-15 03:12:36.007126755 +0000 UTC m=+234.163740963" Jun 15 03:12:36.287482 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:36.287457 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/35718fb0-c60a-4206-a17f-746409b06664-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-7hkl2\" (UID: \"35718fb0-c60a-4206-a17f-746409b06664\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7hkl2" Jun 15 03:12:36.289740 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:36.289719 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/35718fb0-c60a-4206-a17f-746409b06664-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-7hkl2\" (UID: \"35718fb0-c60a-4206-a17f-746409b06664\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7hkl2" Jun 15 03:12:36.579250 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:36.579157 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7hkl2" Jun 15 03:12:36.696396 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:36.696369 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7hkl2"] Jun 15 03:12:36.698717 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:12:36.698688 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35718fb0_c60a_4206_a17f_746409b06664.slice/crio-7b38f701199a4a5c18ec6347b9778d718a94ca5c57ae2f048adc311615c2d381 WatchSource:0}: Error finding container 7b38f701199a4a5c18ec6347b9778d718a94ca5c57ae2f048adc311615c2d381: Status 404 returned error can't find the container with id 7b38f701199a4a5c18ec6347b9778d718a94ca5c57ae2f048adc311615c2d381 Jun 15 03:12:36.982551 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:36.982460 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7hkl2" event={"ID":"35718fb0-c60a-4206-a17f-746409b06664","Type":"ContainerStarted","Data":"7b38f701199a4a5c18ec6347b9778d718a94ca5c57ae2f048adc311615c2d381"} Jun 15 03:12:37.498061 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:37.498024 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:37.500726 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:37.500697 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmv4z\" (UID: \"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307\") " pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:37.723394 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:37.723361 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qmv4z" Jun 15 03:12:37.917252 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:37.917210 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qmv4z"] Jun 15 03:12:37.920645 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:12:37.920617 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce2fb1f_f3b9_4fca_a5ff_d7e43b11d307.slice/crio-d169d1d4bb29c54434b7dadcd1f47a9c7d864c27478a897d24671f2d7eddeaa4 WatchSource:0}: Error finding container d169d1d4bb29c54434b7dadcd1f47a9c7d864c27478a897d24671f2d7eddeaa4: Status 404 returned error can't find the container with id d169d1d4bb29c54434b7dadcd1f47a9c7d864c27478a897d24671f2d7eddeaa4 Jun 15 03:12:37.985873 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:37.985843 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qmv4z" event={"ID":"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307","Type":"ContainerStarted","Data":"d169d1d4bb29c54434b7dadcd1f47a9c7d864c27478a897d24671f2d7eddeaa4"} Jun 15 03:12:37.987225 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:37.987203 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7hkl2" event={"ID":"35718fb0-c60a-4206-a17f-746409b06664","Type":"ContainerStarted","Data":"76c4892419b77249a178b239a3759c3bea475c79b76bcc24aa94c323dd0a1946"} Jun 15 03:12:37.987423 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:37.987394 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7hkl2" Jun 15 03:12:37.992054 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:37.992034 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7hkl2" Jun 15 03:12:38.012974 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.012930 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7hkl2" podStartSLOduration=1.883931196 podStartE2EDuration="3.012915918s" totalImagesPullingTime="1.128984722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:38.012691078 +0000 UTC m=+236.169305286" watchObservedRunningTime="2026-06-15 03:12:38.012915918 +0000 UTC m=+236.169530104" Jun 15 03:12:38.021380 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.021352 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75f868fb85-zr4kw"] Jun 15 03:12:38.024127 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.024113 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.026619 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.026589 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 15 03:12:38.026733 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.026623 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 15 03:12:38.026863 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.026844 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 15 03:12:38.026973 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.026921 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 15 03:12:38.026973 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.026926 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 15 03:12:38.027127 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.026974 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-892n6\"" Jun 15 03:12:38.027127 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.026980 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 15 03:12:38.027127 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.026980 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 15 03:12:38.032774 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.032753 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75f868fb85-zr4kw"] Jun 15 03:12:38.103716 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.103639 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-service-ca\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.103716 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.103674 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-oauth-serving-cert\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.103891 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.103733 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50cab205-34d0-446e-8506-5fee00b73234-console-oauth-config\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.103891 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.103767 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmmhf\" (UniqueName: \"kubernetes.io/projected/50cab205-34d0-446e-8506-5fee00b73234-kube-api-access-qmmhf\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.103891 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.103829 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-console-config\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.103891 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.103860 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50cab205-34d0-446e-8506-5fee00b73234-console-serving-cert\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.204310 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.204271 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-console-config\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.204469 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.204318 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50cab205-34d0-446e-8506-5fee00b73234-console-serving-cert\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.204469 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.204388 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-service-ca\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.204469 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.204411 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-oauth-serving-cert\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.204636 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.204471 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50cab205-34d0-446e-8506-5fee00b73234-console-oauth-config\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.204636 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.204496 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmmhf\" (UniqueName: \"kubernetes.io/projected/50cab205-34d0-446e-8506-5fee00b73234-kube-api-access-qmmhf\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.205130 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.205078 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-oauth-serving-cert\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.205130 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.205124 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-service-ca\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.205302 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.205078 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-console-config\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.206823 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.206803 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50cab205-34d0-446e-8506-5fee00b73234-console-serving-cert\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.206880 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.206819 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50cab205-34d0-446e-8506-5fee00b73234-console-oauth-config\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.212170 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.212153 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmmhf\" (UniqueName: \"kubernetes.io/projected/50cab205-34d0-446e-8506-5fee00b73234-kube-api-access-qmmhf\") pod \"console-75f868fb85-zr4kw\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.333802 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.333522 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:38.498838 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.498679 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75f868fb85-zr4kw"] Jun 15 03:12:38.501598 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:12:38.501567 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50cab205_34d0_446e_8506_5fee00b73234.slice/crio-4d0421a19a05b1b24b39a2d608070d2ca6afcfe980f9cb9d6884680e434d92ce WatchSource:0}: Error finding container 4d0421a19a05b1b24b39a2d608070d2ca6afcfe980f9cb9d6884680e434d92ce: Status 404 returned error can't find the container with id 4d0421a19a05b1b24b39a2d608070d2ca6afcfe980f9cb9d6884680e434d92ce Jun 15 03:12:38.731915 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.731650 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj"] Jun 15 03:12:38.735128 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.735104 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" Jun 15 03:12:38.737744 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.737721 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 15 03:12:38.739296 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.739164 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-c6s9f\"" Jun 15 03:12:38.739296 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.739178 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 15 03:12:38.739462 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.739445 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 15 03:12:38.743312 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.743291 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj"] Jun 15 03:12:38.812489 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.812453 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a737eeb8-bcbb-46db-952b-058c1a22710b-prometheus-operator-tls\") pod \"prometheus-operator-864fcfc6f9-7pmwj\" (UID: \"a737eeb8-bcbb-46db-952b-058c1a22710b\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" Jun 15 03:12:38.812707 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.812505 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a737eeb8-bcbb-46db-952b-058c1a22710b-metrics-client-ca\") pod \"prometheus-operator-864fcfc6f9-7pmwj\" (UID: \"a737eeb8-bcbb-46db-952b-058c1a22710b\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" Jun 15 03:12:38.812707 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.812604 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a737eeb8-bcbb-46db-952b-058c1a22710b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-864fcfc6f9-7pmwj\" (UID: \"a737eeb8-bcbb-46db-952b-058c1a22710b\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" Jun 15 03:12:38.812707 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.812640 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjlzj\" (UniqueName: \"kubernetes.io/projected/a737eeb8-bcbb-46db-952b-058c1a22710b-kube-api-access-qjlzj\") pod \"prometheus-operator-864fcfc6f9-7pmwj\" (UID: \"a737eeb8-bcbb-46db-952b-058c1a22710b\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" Jun 15 03:12:38.913741 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.913701 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a737eeb8-bcbb-46db-952b-058c1a22710b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-864fcfc6f9-7pmwj\" (UID: \"a737eeb8-bcbb-46db-952b-058c1a22710b\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" Jun 15 03:12:38.913741 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.913747 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjlzj\" (UniqueName: \"kubernetes.io/projected/a737eeb8-bcbb-46db-952b-058c1a22710b-kube-api-access-qjlzj\") pod \"prometheus-operator-864fcfc6f9-7pmwj\" (UID: \"a737eeb8-bcbb-46db-952b-058c1a22710b\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" Jun 15 03:12:38.913991 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.913847 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a737eeb8-bcbb-46db-952b-058c1a22710b-prometheus-operator-tls\") pod \"prometheus-operator-864fcfc6f9-7pmwj\" (UID: \"a737eeb8-bcbb-46db-952b-058c1a22710b\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" Jun 15 03:12:38.913991 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.913877 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a737eeb8-bcbb-46db-952b-058c1a22710b-metrics-client-ca\") pod \"prometheus-operator-864fcfc6f9-7pmwj\" (UID: \"a737eeb8-bcbb-46db-952b-058c1a22710b\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" Jun 15 03:12:38.914660 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.914633 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a737eeb8-bcbb-46db-952b-058c1a22710b-metrics-client-ca\") pod \"prometheus-operator-864fcfc6f9-7pmwj\" (UID: \"a737eeb8-bcbb-46db-952b-058c1a22710b\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" Jun 15 03:12:38.916366 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.916340 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a737eeb8-bcbb-46db-952b-058c1a22710b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-864fcfc6f9-7pmwj\" (UID: \"a737eeb8-bcbb-46db-952b-058c1a22710b\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" Jun 15 03:12:38.916456 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.916408 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a737eeb8-bcbb-46db-952b-058c1a22710b-prometheus-operator-tls\") pod \"prometheus-operator-864fcfc6f9-7pmwj\" (UID: \"a737eeb8-bcbb-46db-952b-058c1a22710b\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" Jun 15 03:12:38.923184 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.923160 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjlzj\" (UniqueName: \"kubernetes.io/projected/a737eeb8-bcbb-46db-952b-058c1a22710b-kube-api-access-qjlzj\") pod \"prometheus-operator-864fcfc6f9-7pmwj\" (UID: \"a737eeb8-bcbb-46db-952b-058c1a22710b\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" Jun 15 03:12:38.992893 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:38.992816 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75f868fb85-zr4kw" event={"ID":"50cab205-34d0-446e-8506-5fee00b73234","Type":"ContainerStarted","Data":"4d0421a19a05b1b24b39a2d608070d2ca6afcfe980f9cb9d6884680e434d92ce"} Jun 15 03:12:39.048549 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:39.048515 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" Jun 15 03:12:39.199585 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:39.199550 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj"] Jun 15 03:12:39.202958 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:12:39.202918 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda737eeb8_bcbb_46db_952b_058c1a22710b.slice/crio-ce8cc4bf04236b0cde8761ee6f2ad675fb2fee3654c75b4b5129ba9f357686b8 WatchSource:0}: Error finding container ce8cc4bf04236b0cde8761ee6f2ad675fb2fee3654c75b4b5129ba9f357686b8: Status 404 returned error can't find the container with id ce8cc4bf04236b0cde8761ee6f2ad675fb2fee3654c75b4b5129ba9f357686b8 Jun 15 03:12:39.998127 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:39.998072 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" event={"ID":"a737eeb8-bcbb-46db-952b-058c1a22710b","Type":"ContainerStarted","Data":"ce8cc4bf04236b0cde8761ee6f2ad675fb2fee3654c75b4b5129ba9f357686b8"} Jun 15 03:12:40.613313 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.613275 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d66779b5c-gwj2l"] Jun 15 03:12:40.651611 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.651494 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d66779b5c-gwj2l"] Jun 15 03:12:40.651785 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.651622 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.664004 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.663979 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 15 03:12:40.729510 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.729475 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-service-ca\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.729679 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.729530 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-config\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.729679 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.729560 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-oauth-serving-cert\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.729679 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.729597 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-trusted-ca-bundle\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.729823 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.729702 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-oauth-config\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.729823 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.729763 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-serving-cert\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.729823 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.729790 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klhcc\" (UniqueName: \"kubernetes.io/projected/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-kube-api-access-klhcc\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.830390 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.830359 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-oauth-config\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.830562 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.830405 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-serving-cert\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.830562 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.830425 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klhcc\" (UniqueName: \"kubernetes.io/projected/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-kube-api-access-klhcc\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.830562 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.830484 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-service-ca\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.830562 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.830520 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-config\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.830562 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.830544 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-oauth-serving-cert\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.830823 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.830570 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-trusted-ca-bundle\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.831451 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.831420 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-config\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.831565 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.831449 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-oauth-serving-cert\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.831565 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.831460 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-service-ca\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.831983 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.831958 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-trusted-ca-bundle\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.833005 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.832983 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-oauth-config\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.833005 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.833000 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-serving-cert\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.838327 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.838309 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klhcc\" (UniqueName: \"kubernetes.io/projected/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-kube-api-access-klhcc\") pod \"console-5d66779b5c-gwj2l\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:40.964737 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:40.964649 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:42.706592 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:42.706553 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d66779b5c-gwj2l"] Jun 15 03:12:42.944710 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:12:42.944644 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3c1a233_e7a1_4c5f_be86_5c167b56fb73.slice/crio-83fc41084710ac1c1bb4970533cd355662c7c4b608a590dee72591a22579623d WatchSource:0}: Error finding container 83fc41084710ac1c1bb4970533cd355662c7c4b608a590dee72591a22579623d: Status 404 returned error can't find the container with id 83fc41084710ac1c1bb4970533cd355662c7c4b608a590dee72591a22579623d Jun 15 03:12:43.011545 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:43.011519 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75f868fb85-zr4kw" event={"ID":"50cab205-34d0-446e-8506-5fee00b73234","Type":"ContainerStarted","Data":"839cf8332861f1f69ccc8b3c4f5d2a85d97bf8e99b6f84b15515e51eb3e375dd"} Jun 15 03:12:43.013391 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:43.013365 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d66779b5c-gwj2l" event={"ID":"e3c1a233-e7a1-4c5f-be86-5c167b56fb73","Type":"ContainerStarted","Data":"feadeb0c6e1fe384756deb089c744afc87b28b345f3ed0e89bfea40a439aac52"} Jun 15 03:12:43.013494 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:43.013411 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d66779b5c-gwj2l" event={"ID":"e3c1a233-e7a1-4c5f-be86-5c167b56fb73","Type":"ContainerStarted","Data":"83fc41084710ac1c1bb4970533cd355662c7c4b608a590dee72591a22579623d"} Jun 15 03:12:43.027558 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:43.027516 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75f868fb85-zr4kw" podStartSLOduration=0.953598027 podStartE2EDuration="5.027501109s" totalImagesPullingTime="4.073903082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:43.026707525 +0000 UTC m=+241.183321733" watchObservedRunningTime="2026-06-15 03:12:43.027501109 +0000 UTC m=+241.184115368" Jun 15 03:12:43.051414 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:43.051341 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d66779b5c-gwj2l" podStartSLOduration=3.05132285 podStartE2EDuration="3.05132285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:43.050230372 +0000 UTC m=+241.206844583" watchObservedRunningTime="2026-06-15 03:12:43.05132285 +0000 UTC m=+241.207937060" Jun 15 03:12:44.019949 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:44.019862 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" event={"ID":"a737eeb8-bcbb-46db-952b-058c1a22710b","Type":"ContainerStarted","Data":"60f2fe778d4c7e2e5358874964c3080c523c586feb353ae0d7254a9ecd112fe3"} Jun 15 03:12:44.019949 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:44.019933 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" event={"ID":"a737eeb8-bcbb-46db-952b-058c1a22710b","Type":"ContainerStarted","Data":"fc00a1565150783c3a9dfd0562faf3ab88dff522f9ea5fecd496a81f8e0fa379"} Jun 15 03:12:44.021772 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:44.021741 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qmv4z" event={"ID":"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307","Type":"ContainerStarted","Data":"f62cd77ca06754fde9413413adf6c2ed9dba94875492c4cccbb86b602b10d8f0"} Jun 15 03:12:44.021921 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:44.021798 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qmv4z" event={"ID":"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307","Type":"ContainerStarted","Data":"59fa2c0a8d942de1d79ccafdb1dc368e9768df58e801e25a13d96c507ea2bf0e"} Jun 15 03:12:44.036688 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:44.036644 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-7pmwj" podStartSLOduration=2.247295664 podStartE2EDuration="6.036626922s" totalImagesPullingTime="3.789331258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:44.036206851 +0000 UTC m=+242.192821059" watchObservedRunningTime="2026-06-15 03:12:44.036626922 +0000 UTC m=+242.193241130" Jun 15 03:12:46.032902 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.032863 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qmv4z" event={"ID":"2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307","Type":"ContainerStarted","Data":"e76f7e5f98285587508ff9ab86de1b186c8008d7259ba95dd3ba48b52a07bde2"} Jun 15 03:12:46.053684 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.053636 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qmv4z" podStartSLOduration=33.939943571 podStartE2EDuration="41.053623133s" totalImagesPullingTime="7.113679562s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:12:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:46.053361409 +0000 UTC m=+244.209975637" watchObservedRunningTime="2026-06-15 03:12:46.053623133 +0000 UTC m=+244.210237340" Jun 15 03:12:46.099960 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.099930 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bsdj7"] Jun 15 03:12:46.103241 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.103226 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.105944 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.105922 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 15 03:12:46.106041 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.105973 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-przxp\"" Jun 15 03:12:46.106041 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.106010 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 15 03:12:46.106187 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.106107 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 15 03:12:46.178450 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.178408 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-tls\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.178450 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.178451 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-wtmp\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.178633 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.178483 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18a538b6-74bf-4c7a-a451-3dda824187cb-sys\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.178633 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.178545 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.178633 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.178580 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/18a538b6-74bf-4c7a-a451-3dda824187cb-root\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.178633 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.178618 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v89f2\" (UniqueName: \"kubernetes.io/projected/18a538b6-74bf-4c7a-a451-3dda824187cb-kube-api-access-v89f2\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.178786 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.178640 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18a538b6-74bf-4c7a-a451-3dda824187cb-metrics-client-ca\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.178786 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.178691 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.178786 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.178733 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-textfile\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.279594 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.279564 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v89f2\" (UniqueName: \"kubernetes.io/projected/18a538b6-74bf-4c7a-a451-3dda824187cb-kube-api-access-v89f2\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.279594 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.279597 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18a538b6-74bf-4c7a-a451-3dda824187cb-metrics-client-ca\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.279788 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.279615 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.279788 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.279651 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-textfile\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.279788 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.279710 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-tls\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.279788 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.279736 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-wtmp\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.279788 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.279764 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18a538b6-74bf-4c7a-a451-3dda824187cb-sys\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.280008 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.279790 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.280008 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.279816 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/18a538b6-74bf-4c7a-a451-3dda824187cb-root\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.280008 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:46.279835 2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 15 03:12:46.280008 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:12:46.279896 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-tls podName:18a538b6-74bf-4c7a-a451-3dda824187cb nodeName:}" failed. No retries permitted until 2026-06-15 03:12:46.779877242 +0000 UTC m=+244.936491429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-tls") pod "node-exporter-bsdj7" (UID: "18a538b6-74bf-4c7a-a451-3dda824187cb") : secret "node-exporter-tls" not found Jun 15 03:12:46.280008 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.279910 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/18a538b6-74bf-4c7a-a451-3dda824187cb-root\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.280008 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.279912 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-wtmp\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.280008 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.279937 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18a538b6-74bf-4c7a-a451-3dda824187cb-sys\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.280247 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.280062 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-textfile\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.280327 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.280304 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.280458 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.280444 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18a538b6-74bf-4c7a-a451-3dda824187cb-metrics-client-ca\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.281991 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.281970 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.288543 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.288495 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v89f2\" (UniqueName: \"kubernetes.io/projected/18a538b6-74bf-4c7a-a451-3dda824187cb-kube-api-access-v89f2\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.784654 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.784621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-tls\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:46.786884 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:46.786861 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/18a538b6-74bf-4c7a-a451-3dda824187cb-node-exporter-tls\") pod \"node-exporter-bsdj7\" (UID: \"18a538b6-74bf-4c7a-a451-3dda824187cb\") " pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:47.012541 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:47.012507 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bsdj7" Jun 15 03:12:47.024808 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:12:47.024765 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a538b6_74bf_4c7a_a451_3dda824187cb.slice/crio-fcb88e45cd29efb84ea9fc9de818941b0887a1310f153edcc23f6cfa92ddb614 WatchSource:0}: Error finding container fcb88e45cd29efb84ea9fc9de818941b0887a1310f153edcc23f6cfa92ddb614: Status 404 returned error can't find the container with id fcb88e45cd29efb84ea9fc9de818941b0887a1310f153edcc23f6cfa92ddb614 Jun 15 03:12:47.039174 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:47.039109 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bsdj7" event={"ID":"18a538b6-74bf-4c7a-a451-3dda824187cb","Type":"ContainerStarted","Data":"fcb88e45cd29efb84ea9fc9de818941b0887a1310f153edcc23f6cfa92ddb614"} Jun 15 03:12:48.163283 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.163257 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-65c674f96b-vw7b9"] Jun 15 03:12:48.166650 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.166635 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.169344 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.169321 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 15 03:12:48.169465 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.169346 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 15 03:12:48.169465 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.169351 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-f086tjeg3rm0g\"" Jun 15 03:12:48.169465 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.169398 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 15 03:12:48.169465 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.169425 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-tm9fj\"" Jun 15 03:12:48.169603 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.169469 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 15 03:12:48.169603 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.169506 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 15 03:12:48.177877 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.177854 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65c674f96b-vw7b9"] Jun 15 03:12:48.296301 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.296276 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h42qd\" (UniqueName: \"kubernetes.io/projected/e4f343af-f13b-4775-80b0-766f80a17203-kube-api-access-h42qd\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.296438 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.296332 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-thanos-querier-tls\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.296438 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.296353 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.296438 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.296379 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.296438 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.296403 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-grpc-tls\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.296438 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.296437 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.296594 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.296477 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4f343af-f13b-4775-80b0-766f80a17203-metrics-client-ca\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.296594 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.296516 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.334262 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.334219 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:48.334398 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.334281 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:12:48.335636 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.335614 2560 patch_prober.go:28] interesting pod/console-75f868fb85-zr4kw container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.16:8443/health\": dial tcp 10.132.0.16:8443: connect: connection refused" start-of-body= Jun 15 03:12:48.335723 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.335656 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-75f868fb85-zr4kw" podUID="50cab205-34d0-446e-8506-5fee00b73234" containerName="console" probeResult="failure" output="Get \"https://10.132.0.16:8443/health\": dial tcp 10.132.0.16:8443: connect: connection refused" Jun 15 03:12:48.397822 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.397799 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-thanos-querier-tls\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.397911 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.397825 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.397911 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.397850 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.397911 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.397876 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-grpc-tls\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.398159 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.398121 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.398274 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.398199 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4f343af-f13b-4775-80b0-766f80a17203-metrics-client-ca\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.398274 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.398268 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.399704 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.398329 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h42qd\" (UniqueName: \"kubernetes.io/projected/e4f343af-f13b-4775-80b0-766f80a17203-kube-api-access-h42qd\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.399704 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.399591 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4f343af-f13b-4775-80b0-766f80a17203-metrics-client-ca\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.401500 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.401148 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-thanos-querier-tls\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.401500 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.401352 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.402437 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.402386 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.402437 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.402402 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-grpc-tls\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.402592 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.402537 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.406561 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.406534 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e4f343af-f13b-4775-80b0-766f80a17203-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.408092 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.408066 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h42qd\" (UniqueName: \"kubernetes.io/projected/e4f343af-f13b-4775-80b0-766f80a17203-kube-api-access-h42qd\") pod \"thanos-querier-65c674f96b-vw7b9\" (UID: \"e4f343af-f13b-4775-80b0-766f80a17203\") " pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.512775 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.512737 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:48.641289 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:48.641262 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65c674f96b-vw7b9"] Jun 15 03:12:48.644357 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:12:48.644326 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4f343af_f13b_4775_80b0_766f80a17203.slice/crio-d80c0200cdbf4aee201c132a29a44f7ef267a7b094e2212136d078b8d14a2b5d WatchSource:0}: Error finding container d80c0200cdbf4aee201c132a29a44f7ef267a7b094e2212136d078b8d14a2b5d: Status 404 returned error can't find the container with id d80c0200cdbf4aee201c132a29a44f7ef267a7b094e2212136d078b8d14a2b5d Jun 15 03:12:49.046599 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:49.046568 2560 generic.go:356] "Generic (PLEG): container finished" podID="18a538b6-74bf-4c7a-a451-3dda824187cb" containerID="29773b8ba8a972d0a07704e37f430328fc091a6e54086c49277ae51f64b2b9f4" exitCode=0 Jun 15 03:12:49.046765 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:49.046664 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bsdj7" event={"ID":"18a538b6-74bf-4c7a-a451-3dda824187cb","Type":"ContainerDied","Data":"29773b8ba8a972d0a07704e37f430328fc091a6e54086c49277ae51f64b2b9f4"} Jun 15 03:12:49.047752 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:49.047732 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" event={"ID":"e4f343af-f13b-4775-80b0-766f80a17203","Type":"ContainerStarted","Data":"d80c0200cdbf4aee201c132a29a44f7ef267a7b094e2212136d078b8d14a2b5d"} Jun 15 03:12:50.054208 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:50.054175 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bsdj7" event={"ID":"18a538b6-74bf-4c7a-a451-3dda824187cb","Type":"ContainerStarted","Data":"bc85175b23d525722c4d15d3eea65e9cac662868ff3f664dd430cc1f54154275"} Jun 15 03:12:50.054654 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:50.054234 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bsdj7" event={"ID":"18a538b6-74bf-4c7a-a451-3dda824187cb","Type":"ContainerStarted","Data":"03867c619074c152d58326b8a1aa089b80224447e62c9587d094e4145c0a7621"} Jun 15 03:12:50.074717 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:50.074664 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bsdj7" podStartSLOduration=3.020777629 podStartE2EDuration="4.07464539s" totalImagesPullingTime="988.44924ms" totalInitContainerRuntime="65.418521ms" isStatefulPod=true podCreationTimestamp="2026-06-15 03:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:50.073222557 +0000 UTC m=+248.229836768" watchObservedRunningTime="2026-06-15 03:12:50.07464539 +0000 UTC m=+248.231259600" Jun 15 03:12:50.965773 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:50.964777 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:50.965773 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:50.965719 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:12:50.967172 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:50.967140 2560 patch_prober.go:28] interesting pod/console-5d66779b5c-gwj2l container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.18:8443/health\": dial tcp 10.132.0.18:8443: connect: connection refused" start-of-body= Jun 15 03:12:50.967310 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:50.967188 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5d66779b5c-gwj2l" podUID="e3c1a233-e7a1-4c5f-be86-5c167b56fb73" containerName="console" probeResult="failure" output="Get \"https://10.132.0.18:8443/health\": dial tcp 10.132.0.18:8443: connect: connection refused" Jun 15 03:12:51.060581 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.060540 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" event={"ID":"e4f343af-f13b-4775-80b0-766f80a17203","Type":"ContainerStarted","Data":"51bb506633d725b989a211ed8482f9a4fcbf7a6b605effb87b16f787eb3eb61a"} Jun 15 03:12:51.060913 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.060620 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" event={"ID":"e4f343af-f13b-4775-80b0-766f80a17203","Type":"ContainerStarted","Data":"fb3e92bc2b8c093f0737e1e24fe0dc4f3b91dbd711bc48431367dd7af41d0ddf"} Jun 15 03:12:51.060913 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.060670 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" event={"ID":"e4f343af-f13b-4775-80b0-766f80a17203","Type":"ContainerStarted","Data":"d883fbb5e1fb5731a4dfb7b11d5565ae8197feed9415238a3cdf2f774cb38d16"} Jun 15 03:12:51.228918 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.228840 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75f868fb85-zr4kw"] Jun 15 03:12:51.299107 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.299056 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-769944fdf-sf47j"] Jun 15 03:12:51.302307 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.302286 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.305413 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.305321 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 15 03:12:51.305413 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.305335 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 15 03:12:51.305675 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.305436 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 15 03:12:51.305675 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.305462 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 15 03:12:51.305822 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.305750 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pjlbr\"" Jun 15 03:12:51.305881 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.305850 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 15 03:12:51.313747 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.313728 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 15 03:12:51.319228 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.319203 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-769944fdf-sf47j"] Jun 15 03:12:51.428038 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.428002 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58d07ff6-e115-4085-9eea-ad6bb1baf893-metrics-client-ca\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.428038 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.428045 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/58d07ff6-e115-4085-9eea-ad6bb1baf893-federate-client-tls\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.428268 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.428063 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5l9f\" (UniqueName: \"kubernetes.io/projected/58d07ff6-e115-4085-9eea-ad6bb1baf893-kube-api-access-p5l9f\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.428268 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.428194 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d07ff6-e115-4085-9eea-ad6bb1baf893-serving-certs-ca-bundle\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.428268 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.428241 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d07ff6-e115-4085-9eea-ad6bb1baf893-telemeter-trusted-ca-bundle\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.428429 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.428271 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58d07ff6-e115-4085-9eea-ad6bb1baf893-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.428429 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.428328 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/58d07ff6-e115-4085-9eea-ad6bb1baf893-secret-telemeter-client\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.428429 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.428369 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/58d07ff6-e115-4085-9eea-ad6bb1baf893-telemeter-client-tls\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.528782 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.528761 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58d07ff6-e115-4085-9eea-ad6bb1baf893-metrics-client-ca\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.528851 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.528797 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/58d07ff6-e115-4085-9eea-ad6bb1baf893-federate-client-tls\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.528851 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.528813 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5l9f\" (UniqueName: \"kubernetes.io/projected/58d07ff6-e115-4085-9eea-ad6bb1baf893-kube-api-access-p5l9f\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.528935 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.528880 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d07ff6-e115-4085-9eea-ad6bb1baf893-serving-certs-ca-bundle\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.528935 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.528907 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d07ff6-e115-4085-9eea-ad6bb1baf893-telemeter-trusted-ca-bundle\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.529034 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.528931 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58d07ff6-e115-4085-9eea-ad6bb1baf893-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.529034 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.528974 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/58d07ff6-e115-4085-9eea-ad6bb1baf893-secret-telemeter-client\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.529034 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.529014 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/58d07ff6-e115-4085-9eea-ad6bb1baf893-telemeter-client-tls\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.529686 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.529660 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d07ff6-e115-4085-9eea-ad6bb1baf893-serving-certs-ca-bundle\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.529928 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.529907 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d07ff6-e115-4085-9eea-ad6bb1baf893-telemeter-trusted-ca-bundle\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.530198 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.530179 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58d07ff6-e115-4085-9eea-ad6bb1baf893-metrics-client-ca\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.531774 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.531737 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/58d07ff6-e115-4085-9eea-ad6bb1baf893-federate-client-tls\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.531949 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.531909 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58d07ff6-e115-4085-9eea-ad6bb1baf893-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.531949 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.531918 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/58d07ff6-e115-4085-9eea-ad6bb1baf893-secret-telemeter-client\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.532027 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.531953 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/58d07ff6-e115-4085-9eea-ad6bb1baf893-telemeter-client-tls\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.536210 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.536189 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5l9f\" (UniqueName: \"kubernetes.io/projected/58d07ff6-e115-4085-9eea-ad6bb1baf893-kube-api-access-p5l9f\") pod \"telemeter-client-769944fdf-sf47j\" (UID: \"58d07ff6-e115-4085-9eea-ad6bb1baf893\") " pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.613433 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.613290 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" Jun 15 03:12:51.749016 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:51.748937 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-769944fdf-sf47j"] Jun 15 03:12:51.752247 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:12:51.752209 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58d07ff6_e115_4085_9eea_ad6bb1baf893.slice/crio-0e9a8658267f8aad145364d6a63d204a153c4ceb6ec27a8966379496db8addd9 WatchSource:0}: Error finding container 0e9a8658267f8aad145364d6a63d204a153c4ceb6ec27a8966379496db8addd9: Status 404 returned error can't find the container with id 0e9a8658267f8aad145364d6a63d204a153c4ceb6ec27a8966379496db8addd9 Jun 15 03:12:52.067176 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:52.067136 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" event={"ID":"e4f343af-f13b-4775-80b0-766f80a17203","Type":"ContainerStarted","Data":"eaeadeeca2a14230a0b327d136f39fc00e4712119b350bcb3ec1c810ecb95bc0"} Jun 15 03:12:52.067623 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:52.067229 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" event={"ID":"e4f343af-f13b-4775-80b0-766f80a17203","Type":"ContainerStarted","Data":"372fa348568aaf7be6964c5b3530444fd3f2d3b777546949482f05bc72667806"} Jun 15 03:12:52.067623 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:52.067275 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" event={"ID":"e4f343af-f13b-4775-80b0-766f80a17203","Type":"ContainerStarted","Data":"0aa76f5f4618e53ea0d833381db01bc47b6881d6b42ff5ee108c57a01eb82374"} Jun 15 03:12:52.067623 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:52.067320 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:12:52.068498 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:52.068471 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" event={"ID":"58d07ff6-e115-4085-9eea-ad6bb1baf893","Type":"ContainerStarted","Data":"0e9a8658267f8aad145364d6a63d204a153c4ceb6ec27a8966379496db8addd9"} Jun 15 03:12:52.088704 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:52.088639 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" podStartSLOduration=1.465769277 podStartE2EDuration="4.088623976s" totalImagesPullingTime="2.622854699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:52.08786078 +0000 UTC m=+250.244474990" watchObservedRunningTime="2026-06-15 03:12:52.088623976 +0000 UTC m=+250.245238183" Jun 15 03:12:54.054783 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:54.054752 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs\") pod \"network-metrics-daemon-9z4f4\" (UID: \"ff14c0e9-ddf7-4720-ac87-54a88354948e\") " pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:12:54.057334 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:54.057310 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff14c0e9-ddf7-4720-ac87-54a88354948e-metrics-certs\") pod \"network-metrics-daemon-9z4f4\" (UID: \"ff14c0e9-ddf7-4720-ac87-54a88354948e\") " pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:12:54.077356 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:54.077322 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" event={"ID":"58d07ff6-e115-4085-9eea-ad6bb1baf893","Type":"ContainerStarted","Data":"8ec17cce2968ec77b5b6368caae221b3e4055b25a7d887486daa0ad2633eb937"} Jun 15 03:12:54.265565 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:54.265532 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lnpms\"" Jun 15 03:12:54.272863 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:54.272824 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9z4f4" Jun 15 03:12:54.560999 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:54.560972 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9z4f4"] Jun 15 03:12:54.563037 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:12:54.563005 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff14c0e9_ddf7_4720_ac87_54a88354948e.slice/crio-81e1f6baa395a47bc05ef5281eda817ec0aae3dd7405a23d0795a638a31a56fd WatchSource:0}: Error finding container 81e1f6baa395a47bc05ef5281eda817ec0aae3dd7405a23d0795a638a31a56fd: Status 404 returned error can't find the container with id 81e1f6baa395a47bc05ef5281eda817ec0aae3dd7405a23d0795a638a31a56fd Jun 15 03:12:55.083643 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:55.083608 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" event={"ID":"58d07ff6-e115-4085-9eea-ad6bb1baf893","Type":"ContainerStarted","Data":"a591b2eb496922548c3f5aa02c1eeebba942ca0dac6acdbe56724f603d3a2fd2"} Jun 15 03:12:55.084124 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:55.083674 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" event={"ID":"58d07ff6-e115-4085-9eea-ad6bb1baf893","Type":"ContainerStarted","Data":"b9bba9eb129bdb97c0820d2020fe29a65a45d07887dd01769b7247a202cee3ff"} Jun 15 03:12:55.084911 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:55.084880 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9z4f4" event={"ID":"ff14c0e9-ddf7-4720-ac87-54a88354948e","Type":"ContainerStarted","Data":"81e1f6baa395a47bc05ef5281eda817ec0aae3dd7405a23d0795a638a31a56fd"} Jun 15 03:12:55.107169 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:55.107124 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-769944fdf-sf47j" podStartSLOduration=1.429007654 podStartE2EDuration="4.107107416s" totalImagesPullingTime="2.678099762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:55.104627196 +0000 UTC m=+253.261241404" watchObservedRunningTime="2026-06-15 03:12:55.107107416 +0000 UTC m=+253.263721619" Jun 15 03:12:55.198312 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:55.198284 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d66779b5c-gwj2l"] Jun 15 03:12:56.090779 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:56.090745 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9z4f4" event={"ID":"ff14c0e9-ddf7-4720-ac87-54a88354948e","Type":"ContainerStarted","Data":"4d4ad7a28cc03f53c591954185df458e77a5278b71a2298363a61085fca6c4d3"} Jun 15 03:12:56.091165 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:56.090803 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9z4f4" event={"ID":"ff14c0e9-ddf7-4720-ac87-54a88354948e","Type":"ContainerStarted","Data":"65dc7a5e2e18c2e39dbf3505dc73a8a008296add3926d2aecdc073457cc2ad91"} Jun 15 03:12:56.106707 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:56.106666 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9z4f4" podStartSLOduration=253.170198219 podStartE2EDuration="4m14.106653536s" totalImagesPullingTime="936.455317ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:56.105680162 +0000 UTC m=+254.262294370" watchObservedRunningTime="2026-06-15 03:12:56.106653536 +0000 UTC m=+254.263267744" Jun 15 03:12:58.079153 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:12:58.079121 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-65c674f96b-vw7b9" Jun 15 03:13:16.248221 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:16.248186 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-75f868fb85-zr4kw" podUID="50cab205-34d0-446e-8506-5fee00b73234" containerName="console" containerID="cri-o://839cf8332861f1f69ccc8b3c4f5d2a85d97bf8e99b6f84b15515e51eb3e375dd" gracePeriod=15 Jun 15 03:13:20.219130 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:20.219047 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5d66779b5c-gwj2l" podUID="e3c1a233-e7a1-4c5f-be86-5c167b56fb73" containerName="console" containerID="cri-o://feadeb0c6e1fe384756deb089c744afc87b28b345f3ed0e89bfea40a439aac52" gracePeriod=15 Jun 15 03:13:22.776188 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:13:22.776150 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" podUID="50f26ba5-125d-4c08-bc95-0fef32a874f6" Jun 15 03:13:22.776188 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:13:22.776150 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5zmqd" podUID="f10bedca-8f8a-4db1-9b58-8d3d854ece5d" Jun 15 03:13:23.189305 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:23.189276 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:13:23.189305 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:23.189314 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zmqd" Jun 15 03:13:24.255542 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:24.255514 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-f9d7df769-9mvkb_cde73dc3-0cfa-4b1b-afe1-296bdf4e026c/cluster-monitoring-operator/0.log" Jun 15 03:13:25.454291 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:25.454262 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bsdj7_18a538b6-74bf-4c7a-a451-3dda824187cb/init-textfile/0.log" Jun 15 03:13:25.654526 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:25.654496 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bsdj7_18a538b6-74bf-4c7a-a451-3dda824187cb/node-exporter/0.log" Jun 15 03:13:25.854601 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:25.854576 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bsdj7_18a538b6-74bf-4c7a-a451-3dda824187cb/kube-rbac-proxy/0.log" Jun 15 03:13:26.732346 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:26.732314 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:13:26.732705 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:26.732352 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:13:26.734687 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:26.734669 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f10bedca-8f8a-4db1-9b58-8d3d854ece5d-metrics-tls\") pod \"dns-default-5zmqd\" (UID: \"f10bedca-8f8a-4db1-9b58-8d3d854ece5d\") " pod="openshift-dns/dns-default-5zmqd" Jun 15 03:13:26.734950 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:26.734927 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls\") pod \"image-registry-6786c7fbc5-8q824\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:13:26.794631 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:26.794606 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4l6gg\"" Jun 15 03:13:26.794756 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:26.794644 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b8jdx\"" Jun 15 03:13:26.800378 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:26.800358 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zmqd" Jun 15 03:13:26.800378 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:26.800370 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:13:26.936191 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:26.936040 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5zmqd"] Jun 15 03:13:26.938187 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:13:26.938162 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf10bedca_8f8a_4db1_9b58_8d3d854ece5d.slice/crio-5dd79a9aacf3c0f0f2996133a4a1cb883153becba2cbbe9978f3bbfd0ed59213 WatchSource:0}: Error finding container 5dd79a9aacf3c0f0f2996133a4a1cb883153becba2cbbe9978f3bbfd0ed59213: Status 404 returned error can't find the container with id 5dd79a9aacf3c0f0f2996133a4a1cb883153becba2cbbe9978f3bbfd0ed59213 Jun 15 03:13:26.953867 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:26.953840 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6786c7fbc5-8q824"] Jun 15 03:13:26.956317 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:13:26.956295 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f26ba5_125d_4c08_bc95_0fef32a874f6.slice/crio-715430184c15a65af78ee47e778bab1157b97dd769ed0d18365553b694a4a754 WatchSource:0}: Error finding container 715430184c15a65af78ee47e778bab1157b97dd769ed0d18365553b694a4a754: Status 404 returned error can't find the container with id 715430184c15a65af78ee47e778bab1157b97dd769ed0d18365553b694a4a754 Jun 15 03:13:27.206628 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:27.206595 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" event={"ID":"50f26ba5-125d-4c08-bc95-0fef32a874f6","Type":"ContainerStarted","Data":"66035ca6a12b63a6ea035c29bc2135d135b383070c01689ee60d8b06df985c29"} Jun 15 03:13:27.206824 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:27.206652 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" event={"ID":"50f26ba5-125d-4c08-bc95-0fef32a874f6","Type":"ContainerStarted","Data":"715430184c15a65af78ee47e778bab1157b97dd769ed0d18365553b694a4a754"} Jun 15 03:13:27.206824 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:27.206702 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:13:27.207871 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:27.207845 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zmqd" event={"ID":"f10bedca-8f8a-4db1-9b58-8d3d854ece5d","Type":"ContainerStarted","Data":"5dd79a9aacf3c0f0f2996133a4a1cb883153becba2cbbe9978f3bbfd0ed59213"} Jun 15 03:13:27.228462 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:27.228417 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" podStartSLOduration=285.22840406 podStartE2EDuration="4m45.22840406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:08:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:13:27.227294795 +0000 UTC m=+285.383908998" watchObservedRunningTime="2026-06-15 03:13:27.22840406 +0000 UTC m=+285.385018268" Jun 15 03:13:29.221984 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:29.221948 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zmqd" event={"ID":"f10bedca-8f8a-4db1-9b58-8d3d854ece5d","Type":"ContainerStarted","Data":"f3f5b927469701a509d712d3751e907c9d0630c9f0c013455833f45ef674df37"} Jun 15 03:13:29.222391 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:29.222021 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zmqd" event={"ID":"f10bedca-8f8a-4db1-9b58-8d3d854ece5d","Type":"ContainerStarted","Data":"c67203476ff2ac55c345d0a79a6e0292845722b2b9a77d6089250c570e7a39aa"} Jun 15 03:13:29.222391 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:29.222093 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5zmqd" Jun 15 03:13:29.240736 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:29.240685 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5zmqd" podStartSLOduration=251.463040409 podStartE2EDuration="4m13.240671195s" totalImagesPullingTime="1.777630786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:13:29.239409552 +0000 UTC m=+287.396023793" watchObservedRunningTime="2026-06-15 03:13:29.240671195 +0000 UTC m=+287.397285403" Jun 15 03:13:29.256193 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:29.256159 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-864fcfc6f9-7pmwj_a737eeb8-bcbb-46db-952b-058c1a22710b/prometheus-operator/0.log" Jun 15 03:13:29.454144 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:29.454119 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-864fcfc6f9-7pmwj_a737eeb8-bcbb-46db-952b-058c1a22710b/kube-rbac-proxy/0.log" Jun 15 03:13:29.654601 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:29.654567 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5cf667749c-7hkl2_35718fb0-c60a-4206-a17f-746409b06664/prometheus-operator-admission-webhook/0.log" Jun 15 03:13:29.854292 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:29.854263 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-769944fdf-sf47j_58d07ff6-e115-4085-9eea-ad6bb1baf893/telemeter-client/0.log" Jun 15 03:13:30.054421 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:30.054398 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-769944fdf-sf47j_58d07ff6-e115-4085-9eea-ad6bb1baf893/reload/0.log" Jun 15 03:13:30.254104 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:30.254060 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-769944fdf-sf47j_58d07ff6-e115-4085-9eea-ad6bb1baf893/kube-rbac-proxy/0.log" Jun 15 03:13:30.454596 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:30.454528 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c674f96b-vw7b9_e4f343af-f13b-4775-80b0-766f80a17203/thanos-query/0.log" Jun 15 03:13:30.654069 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:30.654042 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c674f96b-vw7b9_e4f343af-f13b-4775-80b0-766f80a17203/kube-rbac-proxy-web/0.log" Jun 15 03:13:30.854620 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:30.854592 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c674f96b-vw7b9_e4f343af-f13b-4775-80b0-766f80a17203/kube-rbac-proxy/0.log" Jun 15 03:13:31.053835 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.053808 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c674f96b-vw7b9_e4f343af-f13b-4775-80b0-766f80a17203/prom-label-proxy/0.log" Jun 15 03:13:31.254529 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.254504 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c674f96b-vw7b9_e4f343af-f13b-4775-80b0-766f80a17203/kube-rbac-proxy-rules/0.log" Jun 15 03:13:31.390306 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.390285 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-75f868fb85-zr4kw_50cab205-34d0-446e-8506-5fee00b73234/console/0.log" Jun 15 03:13:31.392586 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.392566 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:13:31.454473 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.454428 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c674f96b-vw7b9_e4f343af-f13b-4775-80b0-766f80a17203/kube-rbac-proxy-metrics/0.log" Jun 15 03:13:31.474645 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.474611 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-oauth-serving-cert\") pod \"50cab205-34d0-446e-8506-5fee00b73234\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " Jun 15 03:13:31.474771 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.474676 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-console-config\") pod \"50cab205-34d0-446e-8506-5fee00b73234\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " Jun 15 03:13:31.474771 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.474711 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmmhf\" (UniqueName: \"kubernetes.io/projected/50cab205-34d0-446e-8506-5fee00b73234-kube-api-access-qmmhf\") pod \"50cab205-34d0-446e-8506-5fee00b73234\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " Jun 15 03:13:31.474771 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.474760 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50cab205-34d0-446e-8506-5fee00b73234-console-serving-cert\") pod \"50cab205-34d0-446e-8506-5fee00b73234\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " Jun 15 03:13:31.474938 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.474877 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-service-ca\") pod \"50cab205-34d0-446e-8506-5fee00b73234\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " Jun 15 03:13:31.475001 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.474952 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50cab205-34d0-446e-8506-5fee00b73234-console-oauth-config\") pod \"50cab205-34d0-446e-8506-5fee00b73234\" (UID: \"50cab205-34d0-446e-8506-5fee00b73234\") " Jun 15 03:13:31.475070 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.475044 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "50cab205-34d0-446e-8506-5fee00b73234" (UID: "50cab205-34d0-446e-8506-5fee00b73234"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:13:31.475160 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.475109 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-console-config" (OuterVolumeSpecName: "console-config") pod "50cab205-34d0-446e-8506-5fee00b73234" (UID: "50cab205-34d0-446e-8506-5fee00b73234"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:13:31.475270 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.475230 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-service-ca" (OuterVolumeSpecName: "service-ca") pod "50cab205-34d0-446e-8506-5fee00b73234" (UID: "50cab205-34d0-446e-8506-5fee00b73234"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:13:31.475270 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.475260 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-oauth-serving-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:13:31.475433 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.475281 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-console-config\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:13:31.476986 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.476962 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50cab205-34d0-446e-8506-5fee00b73234-kube-api-access-qmmhf" (OuterVolumeSpecName: "kube-api-access-qmmhf") pod "50cab205-34d0-446e-8506-5fee00b73234" (UID: "50cab205-34d0-446e-8506-5fee00b73234"). InnerVolumeSpecName "kube-api-access-qmmhf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:13:31.477073 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.476999 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50cab205-34d0-446e-8506-5fee00b73234-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "50cab205-34d0-446e-8506-5fee00b73234" (UID: "50cab205-34d0-446e-8506-5fee00b73234"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:13:31.477138 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.477076 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50cab205-34d0-446e-8506-5fee00b73234-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "50cab205-34d0-446e-8506-5fee00b73234" (UID: "50cab205-34d0-446e-8506-5fee00b73234"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:13:31.576367 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.576328 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50cab205-34d0-446e-8506-5fee00b73234-console-serving-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:13:31.576367 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.576363 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50cab205-34d0-446e-8506-5fee00b73234-service-ca\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:13:31.576554 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.576379 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50cab205-34d0-446e-8506-5fee00b73234-console-oauth-config\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:13:31.576554 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.576395 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmmhf\" (UniqueName: \"kubernetes.io/projected/50cab205-34d0-446e-8506-5fee00b73234-kube-api-access-qmmhf\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:13:31.654821 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:31.654770 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-867b76cb74-n9b2r_cf2910da-c1b3-4520-92be-4e1b96f6d878/networking-console-plugin/0.log" Jun 15 03:13:32.236479 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:32.236449 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-75f868fb85-zr4kw_50cab205-34d0-446e-8506-5fee00b73234/console/0.log" Jun 15 03:13:32.236628 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:32.236487 2560 generic.go:356] "Generic (PLEG): container finished" podID="50cab205-34d0-446e-8506-5fee00b73234" containerID="839cf8332861f1f69ccc8b3c4f5d2a85d97bf8e99b6f84b15515e51eb3e375dd" exitCode=137 Jun 15 03:13:32.236628 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:32.236519 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75f868fb85-zr4kw" event={"ID":"50cab205-34d0-446e-8506-5fee00b73234","Type":"ContainerDied","Data":"839cf8332861f1f69ccc8b3c4f5d2a85d97bf8e99b6f84b15515e51eb3e375dd"} Jun 15 03:13:32.236628 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:32.236538 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75f868fb85-zr4kw" Jun 15 03:13:32.236628 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:32.236569 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75f868fb85-zr4kw" event={"ID":"50cab205-34d0-446e-8506-5fee00b73234","Type":"ContainerDied","Data":"4d0421a19a05b1b24b39a2d608070d2ca6afcfe980f9cb9d6884680e434d92ce"} Jun 15 03:13:32.236628 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:32.236590 2560 scope.go:117] "RemoveContainer" containerID="839cf8332861f1f69ccc8b3c4f5d2a85d97bf8e99b6f84b15515e51eb3e375dd" Jun 15 03:13:32.245769 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:32.245746 2560 scope.go:117] "RemoveContainer" containerID="839cf8332861f1f69ccc8b3c4f5d2a85d97bf8e99b6f84b15515e51eb3e375dd" Jun 15 03:13:32.246007 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:13:32.245986 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"839cf8332861f1f69ccc8b3c4f5d2a85d97bf8e99b6f84b15515e51eb3e375dd\": container with ID starting with 839cf8332861f1f69ccc8b3c4f5d2a85d97bf8e99b6f84b15515e51eb3e375dd not found: ID does not exist" containerID="839cf8332861f1f69ccc8b3c4f5d2a85d97bf8e99b6f84b15515e51eb3e375dd" Jun 15 03:13:32.246076 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:32.246020 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"839cf8332861f1f69ccc8b3c4f5d2a85d97bf8e99b6f84b15515e51eb3e375dd"} err="failed to get container status \"839cf8332861f1f69ccc8b3c4f5d2a85d97bf8e99b6f84b15515e51eb3e375dd\": rpc error: code = NotFound desc = could not find container \"839cf8332861f1f69ccc8b3c4f5d2a85d97bf8e99b6f84b15515e51eb3e375dd\": container with ID starting with 839cf8332861f1f69ccc8b3c4f5d2a85d97bf8e99b6f84b15515e51eb3e375dd not found: ID does not exist" Jun 15 03:13:32.257825 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:32.257801 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75f868fb85-zr4kw"] Jun 15 03:13:32.267532 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:32.267511 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75f868fb85-zr4kw"] Jun 15 03:13:32.454158 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:32.454133 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d66779b5c-gwj2l_e3c1a233-e7a1-4c5f-be86-5c167b56fb73/console/0.log" Jun 15 03:13:33.254383 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:33.254353 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5zmqd_f10bedca-8f8a-4db1-9b58-8d3d854ece5d/dns/0.log" Jun 15 03:13:33.454459 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:33.454426 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5zmqd_f10bedca-8f8a-4db1-9b58-8d3d854ece5d/kube-rbac-proxy/0.log" Jun 15 03:13:34.266394 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:34.266354 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50cab205-34d0-446e-8506-5fee00b73234" path="/var/lib/kubelet/pods/50cab205-34d0-446e-8506-5fee00b73234/volumes" Jun 15 03:13:34.867103 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:34.867065 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zq7r8_48bd8c21-f7a6-47fb-adda-1f4d03b37c64/dns-node-resolver/0.log" Jun 15 03:13:35.055049 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.055021 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6786c7fbc5-8q824_50f26ba5-125d-4c08-bc95-0fef32a874f6/registry/0.log" Jun 15 03:13:35.249459 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.249435 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d66779b5c-gwj2l_e3c1a233-e7a1-4c5f-be86-5c167b56fb73/console/0.log" Jun 15 03:13:35.249577 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.249475 2560 generic.go:356] "Generic (PLEG): container finished" podID="e3c1a233-e7a1-4c5f-be86-5c167b56fb73" containerID="feadeb0c6e1fe384756deb089c744afc87b28b345f3ed0e89bfea40a439aac52" exitCode=137 Jun 15 03:13:35.249577 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.249546 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d66779b5c-gwj2l" event={"ID":"e3c1a233-e7a1-4c5f-be86-5c167b56fb73","Type":"ContainerDied","Data":"feadeb0c6e1fe384756deb089c744afc87b28b345f3ed0e89bfea40a439aac52"} Jun 15 03:13:35.656421 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.656391 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vzjh4_5c55e28b-e913-482a-b2ef-e9bf7d2e78e8/node-ca/0.log" Jun 15 03:13:35.854696 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.854678 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d66779b5c-gwj2l_e3c1a233-e7a1-4c5f-be86-5c167b56fb73/console/0.log" Jun 15 03:13:35.857067 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.857049 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:13:35.927859 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.927791 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-oauth-serving-cert\") pod \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " Jun 15 03:13:35.927859 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.927832 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-service-ca\") pod \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " Jun 15 03:13:35.928331 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.927866 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-oauth-config\") pod \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " Jun 15 03:13:35.928331 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.927895 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-trusted-ca-bundle\") pod \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " Jun 15 03:13:35.928331 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.927927 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-config\") pod \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " Jun 15 03:13:35.928331 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.927952 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klhcc\" (UniqueName: \"kubernetes.io/projected/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-kube-api-access-klhcc\") pod \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " Jun 15 03:13:35.928331 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.927976 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-serving-cert\") pod \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\" (UID: \"e3c1a233-e7a1-4c5f-be86-5c167b56fb73\") " Jun 15 03:13:35.928331 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.928281 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e3c1a233-e7a1-4c5f-be86-5c167b56fb73" (UID: "e3c1a233-e7a1-4c5f-be86-5c167b56fb73"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:13:35.928331 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.928295 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-service-ca" (OuterVolumeSpecName: "service-ca") pod "e3c1a233-e7a1-4c5f-be86-5c167b56fb73" (UID: "e3c1a233-e7a1-4c5f-be86-5c167b56fb73"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:13:35.928673 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.928388 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-config" (OuterVolumeSpecName: "console-config") pod "e3c1a233-e7a1-4c5f-be86-5c167b56fb73" (UID: "e3c1a233-e7a1-4c5f-be86-5c167b56fb73"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:13:35.928673 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.928535 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e3c1a233-e7a1-4c5f-be86-5c167b56fb73" (UID: "e3c1a233-e7a1-4c5f-be86-5c167b56fb73"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:13:35.930225 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.930202 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e3c1a233-e7a1-4c5f-be86-5c167b56fb73" (UID: "e3c1a233-e7a1-4c5f-be86-5c167b56fb73"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:13:35.930225 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.930207 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-kube-api-access-klhcc" (OuterVolumeSpecName: "kube-api-access-klhcc") pod "e3c1a233-e7a1-4c5f-be86-5c167b56fb73" (UID: "e3c1a233-e7a1-4c5f-be86-5c167b56fb73"). InnerVolumeSpecName "kube-api-access-klhcc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:13:35.930331 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:35.930297 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e3c1a233-e7a1-4c5f-be86-5c167b56fb73" (UID: "e3c1a233-e7a1-4c5f-be86-5c167b56fb73"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:13:36.029457 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:36.029419 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-oauth-serving-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:13:36.029457 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:36.029452 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-service-ca\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:13:36.029457 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:36.029462 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-oauth-config\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:13:36.029684 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:36.029471 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-trusted-ca-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:13:36.029684 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:36.029479 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-config\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:13:36.029684 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:36.029487 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klhcc\" (UniqueName: \"kubernetes.io/projected/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-kube-api-access-klhcc\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:13:36.029684 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:36.029495 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3c1a233-e7a1-4c5f-be86-5c167b56fb73-console-serving-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:13:36.054764 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:36.054737 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55f78b5b6d-xfjzj_1429058a-5113-4842-83d5-1f57f1fee24a/router/0.log" Jun 15 03:13:36.254365 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:36.254285 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d66779b5c-gwj2l_e3c1a233-e7a1-4c5f-be86-5c167b56fb73/console/0.log" Jun 15 03:13:36.254534 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:36.254379 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d66779b5c-gwj2l" Jun 15 03:13:36.254534 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:36.254381 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d66779b5c-gwj2l" event={"ID":"e3c1a233-e7a1-4c5f-be86-5c167b56fb73","Type":"ContainerDied","Data":"83fc41084710ac1c1bb4970533cd355662c7c4b608a590dee72591a22579623d"} Jun 15 03:13:36.254534 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:36.254459 2560 scope.go:117] "RemoveContainer" containerID="feadeb0c6e1fe384756deb089c744afc87b28b345f3ed0e89bfea40a439aac52" Jun 15 03:13:36.274839 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:36.274814 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d66779b5c-gwj2l"] Jun 15 03:13:36.280313 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:36.280288 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d66779b5c-gwj2l"] Jun 15 03:13:36.655155 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:36.655127 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s7jcv_5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5/serve-healthcheck-canary/0.log" Jun 15 03:13:38.267193 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:38.267163 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3c1a233-e7a1-4c5f-be86-5c167b56fb73" path="/var/lib/kubelet/pods/e3c1a233-e7a1-4c5f-be86-5c167b56fb73/volumes" Jun 15 03:13:39.231697 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:39.231665 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5zmqd" Jun 15 03:13:42.216036 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:42.216011 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:13:42.216598 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:42.216578 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:13:42.223209 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:42.223193 2560 kubelet.go:1648] "Image garbage collection succeeded" Jun 15 03:13:46.805511 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:46.805479 2560 patch_prober.go:28] interesting pod/image-registry-6786c7fbc5-8q824 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 15 03:13:46.808214 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:46.805538 2560 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" podUID="50f26ba5-125d-4c08-bc95-0fef32a874f6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 15 03:13:48.216931 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:48.216903 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:13:57.006667 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:13:57.006631 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6786c7fbc5-8q824"] Jun 15 03:14:08.257949 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.257911 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 15 03:14:08.258493 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.258368 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50cab205-34d0-446e-8506-5fee00b73234" containerName="console" Jun 15 03:14:08.258493 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.258388 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="50cab205-34d0-446e-8506-5fee00b73234" containerName="console" Jun 15 03:14:08.258493 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.258404 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3c1a233-e7a1-4c5f-be86-5c167b56fb73" containerName="console" Jun 15 03:14:08.258493 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.258413 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3c1a233-e7a1-4c5f-be86-5c167b56fb73" containerName="console" Jun 15 03:14:08.258493 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.258488 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="50cab205-34d0-446e-8506-5fee00b73234" containerName="console" Jun 15 03:14:08.258752 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.258499 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3c1a233-e7a1-4c5f-be86-5c167b56fb73" containerName="console" Jun 15 03:14:08.264425 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.264402 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.267279 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.267257 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2vrpx\"" Jun 15 03:14:08.268136 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.268109 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 15 03:14:08.268136 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.268078 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 15 03:14:08.268343 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.268221 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 15 03:14:08.268343 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.268270 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 15 03:14:08.268456 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.268372 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 15 03:14:08.268639 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.268621 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 15 03:14:08.268778 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.268760 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 15 03:14:08.268879 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.268797 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 15 03:14:08.273275 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.273256 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 15 03:14:08.276673 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.276651 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 15 03:14:08.283209 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.283188 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54f1a04f-7091-4542-bc6b-e1c137c16710-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.283296 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.283244 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.283403 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.283382 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.283449 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.283422 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/54f1a04f-7091-4542-bc6b-e1c137c16710-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.283489 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.283458 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.283539 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.283507 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-web-config\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.283580 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.283541 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/54f1a04f-7091-4542-bc6b-e1c137c16710-config-out\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.283646 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.283630 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/54f1a04f-7091-4542-bc6b-e1c137c16710-tls-assets\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.283699 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.283685 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.283758 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.283711 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf4hq\" (UniqueName: \"kubernetes.io/projected/54f1a04f-7091-4542-bc6b-e1c137c16710-kube-api-access-sf4hq\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.283758 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.283728 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-config-volume\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.283758 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.283752 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.283913 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.283866 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f1a04f-7091-4542-bc6b-e1c137c16710-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.384858 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.384828 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf4hq\" (UniqueName: \"kubernetes.io/projected/54f1a04f-7091-4542-bc6b-e1c137c16710-kube-api-access-sf4hq\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.384858 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.384858 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-config-volume\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.385065 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.384880 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.385065 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.384976 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f1a04f-7091-4542-bc6b-e1c137c16710-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.385065 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.385032 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54f1a04f-7091-4542-bc6b-e1c137c16710-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.385251 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.385095 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.385251 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.385123 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.385251 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.385148 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/54f1a04f-7091-4542-bc6b-e1c137c16710-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.385251 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.385178 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.385251 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.385249 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-web-config\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.385511 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.385279 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/54f1a04f-7091-4542-bc6b-e1c137c16710-config-out\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.385511 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.385306 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/54f1a04f-7091-4542-bc6b-e1c137c16710-tls-assets\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.385511 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.385333 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.385898 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.385873 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54f1a04f-7091-4542-bc6b-e1c137c16710-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.386061 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.385892 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/54f1a04f-7091-4542-bc6b-e1c137c16710-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.386207 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.385972 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f1a04f-7091-4542-bc6b-e1c137c16710-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.388265 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.388245 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-config-volume\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.388559 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.388538 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.388646 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.388557 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.388725 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.388701 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.389064 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.389044 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.389191 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.389108 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/54f1a04f-7091-4542-bc6b-e1c137c16710-tls-assets\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.389253 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.389228 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.389355 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.389334 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/54f1a04f-7091-4542-bc6b-e1c137c16710-web-config\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.389777 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.389759 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/54f1a04f-7091-4542-bc6b-e1c137c16710-config-out\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.393098 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.393065 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf4hq\" (UniqueName: \"kubernetes.io/projected/54f1a04f-7091-4542-bc6b-e1c137c16710-kube-api-access-sf4hq\") pod \"alertmanager-main-0\" (UID: \"54f1a04f-7091-4542-bc6b-e1c137c16710\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.575182 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.575153 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.717642 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:08.717604 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 15 03:14:08.724129 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:14:08.724103 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54f1a04f_7091_4542_bc6b_e1c137c16710.slice/crio-7fa7947585aa331a1e1d2919381fcbe982f89008dfb82f10bd5181c9502e64db WatchSource:0}: Error finding container 7fa7947585aa331a1e1d2919381fcbe982f89008dfb82f10bd5181c9502e64db: Status 404 returned error can't find the container with id 7fa7947585aa331a1e1d2919381fcbe982f89008dfb82f10bd5181c9502e64db Jun 15 03:14:09.383273 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:09.383239 2560 generic.go:356] "Generic (PLEG): container finished" podID="54f1a04f-7091-4542-bc6b-e1c137c16710" containerID="957690e633290f63870e6606dc78693e6715fa702cdbed756809363c625264dc" exitCode=0 Jun 15 03:14:09.383678 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:09.383296 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54f1a04f-7091-4542-bc6b-e1c137c16710","Type":"ContainerDied","Data":"957690e633290f63870e6606dc78693e6715fa702cdbed756809363c625264dc"} Jun 15 03:14:09.383678 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:09.383350 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54f1a04f-7091-4542-bc6b-e1c137c16710","Type":"ContainerStarted","Data":"7fa7947585aa331a1e1d2919381fcbe982f89008dfb82f10bd5181c9502e64db"} Jun 15 03:14:09.384407 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:09.384390 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 03:14:11.393557 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:11.393532 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54f1a04f-7091-4542-bc6b-e1c137c16710","Type":"ContainerStarted","Data":"f1f98f505ea34914276eea161c4b85615b3f9cd5ff333f1c3a06edb791b0d14c"} Jun 15 03:14:11.393835 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:11.393592 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54f1a04f-7091-4542-bc6b-e1c137c16710","Type":"ContainerStarted","Data":"8549a1aa255f0ae2dce4631b07d048918e85cb102ba0e3dd05914c38bdab5441"} Jun 15 03:14:11.393835 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:11.393616 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54f1a04f-7091-4542-bc6b-e1c137c16710","Type":"ContainerStarted","Data":"0f64dc72a841414bce971ae4649338c092b9944067108ebbc96f71982738afb9"} Jun 15 03:14:12.400156 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:12.400118 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54f1a04f-7091-4542-bc6b-e1c137c16710","Type":"ContainerStarted","Data":"814ce09336b2ceca53695236a3ddefef9583f1c45222cf65b8ddb62c4194bd17"} Jun 15 03:14:12.400525 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:12.400194 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54f1a04f-7091-4542-bc6b-e1c137c16710","Type":"ContainerStarted","Data":"122e17fb29816d469608fe5aa5d03bd64fc87b2aeefe7ec0ea1805a2a52402db"} Jun 15 03:14:12.400525 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:12.400231 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"54f1a04f-7091-4542-bc6b-e1c137c16710","Type":"ContainerStarted","Data":"3563c747715b65f8230a6bd0e841b9ff928ead5f2aea63202e8eaa45f93d21ef"} Jun 15 03:14:12.428253 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:12.428202 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.611441065 podStartE2EDuration="4.428187562s" totalImagesPullingTime="1.802787345s" totalInitContainerRuntime="13.959152ms" isStatefulPod=false podCreationTimestamp="2026-06-15 03:14:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:14:12.425747166 +0000 UTC m=+330.582361379" watchObservedRunningTime="2026-06-15 03:14:12.428187562 +0000 UTC m=+330.584801770" Jun 15 03:14:22.026116 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.026014 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" podUID="50f26ba5-125d-4c08-bc95-0fef32a874f6" containerName="registry" containerID="cri-o://66035ca6a12b63a6ea035c29bc2135d135b383070c01689ee60d8b06df985c29" gracePeriod=30 Jun 15 03:14:22.279213 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.279142 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:14:22.407838 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.407811 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5k449\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-kube-api-access-5k449\") pod \"50f26ba5-125d-4c08-bc95-0fef32a874f6\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " Jun 15 03:14:22.408021 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.407851 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls\") pod \"50f26ba5-125d-4c08-bc95-0fef32a874f6\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " Jun 15 03:14:22.408021 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.407882 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50f26ba5-125d-4c08-bc95-0fef32a874f6-image-registry-private-configuration\") pod \"50f26ba5-125d-4c08-bc95-0fef32a874f6\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " Jun 15 03:14:22.408021 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.407925 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50f26ba5-125d-4c08-bc95-0fef32a874f6-trusted-ca\") pod \"50f26ba5-125d-4c08-bc95-0fef32a874f6\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " Jun 15 03:14:22.408021 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.407948 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50f26ba5-125d-4c08-bc95-0fef32a874f6-ca-trust-extracted\") pod \"50f26ba5-125d-4c08-bc95-0fef32a874f6\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " Jun 15 03:14:22.408021 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.407970 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-certificates\") pod \"50f26ba5-125d-4c08-bc95-0fef32a874f6\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " Jun 15 03:14:22.408021 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.408001 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-bound-sa-token\") pod \"50f26ba5-125d-4c08-bc95-0fef32a874f6\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " Jun 15 03:14:22.408355 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.408026 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50f26ba5-125d-4c08-bc95-0fef32a874f6-installation-pull-secrets\") pod \"50f26ba5-125d-4c08-bc95-0fef32a874f6\" (UID: \"50f26ba5-125d-4c08-bc95-0fef32a874f6\") " Jun 15 03:14:22.408447 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.408417 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "50f26ba5-125d-4c08-bc95-0fef32a874f6" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:14:22.408511 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.408441 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50f26ba5-125d-4c08-bc95-0fef32a874f6-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "50f26ba5-125d-4c08-bc95-0fef32a874f6" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:14:22.410387 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.410359 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "50f26ba5-125d-4c08-bc95-0fef32a874f6" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:14:22.410520 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.410464 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50f26ba5-125d-4c08-bc95-0fef32a874f6-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "50f26ba5-125d-4c08-bc95-0fef32a874f6" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:22.410674 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.410640 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-kube-api-access-5k449" (OuterVolumeSpecName: "kube-api-access-5k449") pod "50f26ba5-125d-4c08-bc95-0fef32a874f6" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6"). InnerVolumeSpecName "kube-api-access-5k449". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:14:22.410738 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.410706 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "50f26ba5-125d-4c08-bc95-0fef32a874f6" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:14:22.410798 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.410734 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50f26ba5-125d-4c08-bc95-0fef32a874f6-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "50f26ba5-125d-4c08-bc95-0fef32a874f6" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:22.416759 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.416731 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50f26ba5-125d-4c08-bc95-0fef32a874f6-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "50f26ba5-125d-4c08-bc95-0fef32a874f6" (UID: "50f26ba5-125d-4c08-bc95-0fef32a874f6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:14:22.443747 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.443718 2560 generic.go:356] "Generic (PLEG): container finished" podID="50f26ba5-125d-4c08-bc95-0fef32a874f6" containerID="66035ca6a12b63a6ea035c29bc2135d135b383070c01689ee60d8b06df985c29" exitCode=0 Jun 15 03:14:22.443840 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.443757 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" event={"ID":"50f26ba5-125d-4c08-bc95-0fef32a874f6","Type":"ContainerDied","Data":"66035ca6a12b63a6ea035c29bc2135d135b383070c01689ee60d8b06df985c29"} Jun 15 03:14:22.443840 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.443767 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" Jun 15 03:14:22.443840 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.443805 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6786c7fbc5-8q824" event={"ID":"50f26ba5-125d-4c08-bc95-0fef32a874f6","Type":"ContainerDied","Data":"715430184c15a65af78ee47e778bab1157b97dd769ed0d18365553b694a4a754"} Jun 15 03:14:22.443840 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.443835 2560 scope.go:117] "RemoveContainer" containerID="66035ca6a12b63a6ea035c29bc2135d135b383070c01689ee60d8b06df985c29" Jun 15 03:14:22.451999 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.451982 2560 scope.go:117] "RemoveContainer" containerID="66035ca6a12b63a6ea035c29bc2135d135b383070c01689ee60d8b06df985c29" Jun 15 03:14:22.452252 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:14:22.452233 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66035ca6a12b63a6ea035c29bc2135d135b383070c01689ee60d8b06df985c29\": container with ID starting with 66035ca6a12b63a6ea035c29bc2135d135b383070c01689ee60d8b06df985c29 not found: ID does not exist" containerID="66035ca6a12b63a6ea035c29bc2135d135b383070c01689ee60d8b06df985c29" Jun 15 03:14:22.452303 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.452261 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66035ca6a12b63a6ea035c29bc2135d135b383070c01689ee60d8b06df985c29"} err="failed to get container status \"66035ca6a12b63a6ea035c29bc2135d135b383070c01689ee60d8b06df985c29\": rpc error: code = NotFound desc = could not find container \"66035ca6a12b63a6ea035c29bc2135d135b383070c01689ee60d8b06df985c29\": container with ID starting with 66035ca6a12b63a6ea035c29bc2135d135b383070c01689ee60d8b06df985c29 not found: ID does not exist" Jun 15 03:14:22.465059 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.465037 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6786c7fbc5-8q824"] Jun 15 03:14:22.468250 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.468225 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6786c7fbc5-8q824"] Jun 15 03:14:22.509334 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.509310 2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50f26ba5-125d-4c08-bc95-0fef32a874f6-installation-pull-secrets\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:14:22.509334 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.509332 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5k449\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-kube-api-access-5k449\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:14:22.509471 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.509349 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-tls\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:14:22.509471 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.509362 2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50f26ba5-125d-4c08-bc95-0fef32a874f6-image-registry-private-configuration\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:14:22.509471 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.509376 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50f26ba5-125d-4c08-bc95-0fef32a874f6-trusted-ca\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:14:22.509471 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.509388 2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50f26ba5-125d-4c08-bc95-0fef32a874f6-ca-trust-extracted\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:14:22.509471 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.509399 2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50f26ba5-125d-4c08-bc95-0fef32a874f6-registry-certificates\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:14:22.509471 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:22.509413 2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f26ba5-125d-4c08-bc95-0fef32a874f6-bound-sa-token\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:14:24.266601 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:24.266570 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50f26ba5-125d-4c08-bc95-0fef32a874f6" path="/var/lib/kubelet/pods/50f26ba5-125d-4c08-bc95-0fef32a874f6/volumes" Jun 15 03:14:57.693349 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.693315 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8"] Jun 15 03:14:57.693810 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.693629 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50f26ba5-125d-4c08-bc95-0fef32a874f6" containerName="registry" Jun 15 03:14:57.693810 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.693643 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="50f26ba5-125d-4c08-bc95-0fef32a874f6" containerName="registry" Jun 15 03:14:57.693810 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.693695 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="50f26ba5-125d-4c08-bc95-0fef32a874f6" containerName="registry" Jun 15 03:14:57.696711 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.696696 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" Jun 15 03:14:57.699287 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.699267 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 03:14:57.700623 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.700597 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9h8zm\"" Jun 15 03:14:57.700723 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.700640 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 03:14:57.706890 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.706870 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8"] Jun 15 03:14:57.879431 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.879401 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8\" (UID: \"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" Jun 15 03:14:57.879584 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.879444 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8\" (UID: \"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" Jun 15 03:14:57.879584 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.879475 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c56rb\" (UniqueName: \"kubernetes.io/projected/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-kube-api-access-c56rb\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8\" (UID: \"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" Jun 15 03:14:57.980723 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.980649 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8\" (UID: \"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" Jun 15 03:14:57.980723 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.980700 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8\" (UID: \"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" Jun 15 03:14:57.980871 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.980727 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c56rb\" (UniqueName: \"kubernetes.io/projected/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-kube-api-access-c56rb\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8\" (UID: \"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" Jun 15 03:14:57.981033 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.981014 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8\" (UID: \"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" Jun 15 03:14:57.981071 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.981030 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8\" (UID: \"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" Jun 15 03:14:57.989505 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:57.989474 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c56rb\" (UniqueName: \"kubernetes.io/projected/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-kube-api-access-c56rb\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8\" (UID: \"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" Jun 15 03:14:58.005474 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:58.005447 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" Jun 15 03:14:58.126792 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:58.126768 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8"] Jun 15 03:14:58.128942 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:14:58.128912 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf484a184_4bb6_4c9a_9c2d_4f7d1d23b7a1.slice/crio-8fd5af9ab069b6cde7dc505c10885b52b264820619a22df66d8055b297a1a901 WatchSource:0}: Error finding container 8fd5af9ab069b6cde7dc505c10885b52b264820619a22df66d8055b297a1a901: Status 404 returned error can't find the container with id 8fd5af9ab069b6cde7dc505c10885b52b264820619a22df66d8055b297a1a901 Jun 15 03:14:58.581622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:14:58.581583 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" event={"ID":"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1","Type":"ContainerStarted","Data":"8fd5af9ab069b6cde7dc505c10885b52b264820619a22df66d8055b297a1a901"} Jun 15 03:15:03.603809 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:03.603777 2560 generic.go:356] "Generic (PLEG): container finished" podID="f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1" containerID="1710c3d4a2814efcb6e52ad8644a680945bc5e39eb22037ebd79f3d1eb9538d1" exitCode=0 Jun 15 03:15:03.604220 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:03.603832 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" event={"ID":"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1","Type":"ContainerDied","Data":"1710c3d4a2814efcb6e52ad8644a680945bc5e39eb22037ebd79f3d1eb9538d1"} Jun 15 03:15:06.618058 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:06.618022 2560 generic.go:356] "Generic (PLEG): container finished" podID="f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1" containerID="20d69ab203a7b32e1a4d9986f771e12fdf4c1bd80cd3c30b8e13047d88adcc1c" exitCode=0 Jun 15 03:15:06.618430 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:06.618121 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" event={"ID":"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1","Type":"ContainerDied","Data":"20d69ab203a7b32e1a4d9986f771e12fdf4c1bd80cd3c30b8e13047d88adcc1c"} Jun 15 03:15:13.651958 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:13.651919 2560 generic.go:356] "Generic (PLEG): container finished" podID="f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1" containerID="e49e4a1b6126438f032be9c622a037a030799f4589212fddea86f24fbc61db5f" exitCode=0 Jun 15 03:15:13.652406 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:13.651958 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" event={"ID":"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1","Type":"ContainerDied","Data":"e49e4a1b6126438f032be9c622a037a030799f4589212fddea86f24fbc61db5f"} Jun 15 03:15:14.788267 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:14.788243 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" Jun 15 03:15:14.833039 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:14.833009 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-util\") pod \"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1\" (UID: \"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1\") " Jun 15 03:15:14.833201 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:14.833044 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-bundle\") pod \"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1\" (UID: \"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1\") " Jun 15 03:15:14.833201 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:14.833075 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c56rb\" (UniqueName: \"kubernetes.io/projected/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-kube-api-access-c56rb\") pod \"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1\" (UID: \"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1\") " Jun 15 03:15:14.833814 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:14.833787 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-bundle" (OuterVolumeSpecName: "bundle") pod "f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1" (UID: "f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:15:14.835210 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:14.835186 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-kube-api-access-c56rb" (OuterVolumeSpecName: "kube-api-access-c56rb") pod "f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1" (UID: "f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1"). InnerVolumeSpecName "kube-api-access-c56rb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:15:14.838895 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:14.838871 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-util" (OuterVolumeSpecName: "util") pod "f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1" (UID: "f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:15:14.934721 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:14.934655 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:15:14.934721 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:14.934681 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:15:14.934721 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:14.934690 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c56rb\" (UniqueName: \"kubernetes.io/projected/f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1-kube-api-access-c56rb\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:15:15.661239 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:15.661207 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" Jun 15 03:15:15.661416 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:15.661208 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dhjrg8" event={"ID":"f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1","Type":"ContainerDied","Data":"8fd5af9ab069b6cde7dc505c10885b52b264820619a22df66d8055b297a1a901"} Jun 15 03:15:15.661416 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:15.661351 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8fd5af9ab069b6cde7dc505c10885b52b264820619a22df66d8055b297a1a901" Jun 15 03:15:20.386805 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.386771 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-dtg6w"] Jun 15 03:15:20.387191 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.387057 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1" containerName="extract" Jun 15 03:15:20.387191 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.387067 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1" containerName="extract" Jun 15 03:15:20.387191 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.387093 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1" containerName="util" Jun 15 03:15:20.387191 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.387099 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1" containerName="util" Jun 15 03:15:20.387191 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.387115 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1" containerName="pull" Jun 15 03:15:20.387191 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.387123 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1" containerName="pull" Jun 15 03:15:20.387191 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.387168 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f484a184-4bb6-4c9a-9c2d-4f7d1d23b7a1" containerName="extract" Jun 15 03:15:20.391699 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.391683 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-dtg6w" Jun 15 03:15:20.395752 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.395730 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-gzvhx\"" Jun 15 03:15:20.395752 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.395740 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 15 03:15:20.395922 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.395768 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 15 03:15:20.403259 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.403238 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-dtg6w"] Jun 15 03:15:20.480430 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.480401 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d433be25-44af-4432-b487-c1be830fb4b8-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-dtg6w\" (UID: \"d433be25-44af-4432-b487-c1be830fb4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-dtg6w" Jun 15 03:15:20.480574 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.480459 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czxmq\" (UniqueName: \"kubernetes.io/projected/d433be25-44af-4432-b487-c1be830fb4b8-kube-api-access-czxmq\") pod \"cert-manager-operator-controller-manager-57757f7cfc-dtg6w\" (UID: \"d433be25-44af-4432-b487-c1be830fb4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-dtg6w" Jun 15 03:15:20.581327 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.581295 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czxmq\" (UniqueName: \"kubernetes.io/projected/d433be25-44af-4432-b487-c1be830fb4b8-kube-api-access-czxmq\") pod \"cert-manager-operator-controller-manager-57757f7cfc-dtg6w\" (UID: \"d433be25-44af-4432-b487-c1be830fb4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-dtg6w" Jun 15 03:15:20.581467 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.581347 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d433be25-44af-4432-b487-c1be830fb4b8-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-dtg6w\" (UID: \"d433be25-44af-4432-b487-c1be830fb4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-dtg6w" Jun 15 03:15:20.581658 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.581641 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d433be25-44af-4432-b487-c1be830fb4b8-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-dtg6w\" (UID: \"d433be25-44af-4432-b487-c1be830fb4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-dtg6w" Jun 15 03:15:20.593728 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.593700 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czxmq\" (UniqueName: \"kubernetes.io/projected/d433be25-44af-4432-b487-c1be830fb4b8-kube-api-access-czxmq\") pod \"cert-manager-operator-controller-manager-57757f7cfc-dtg6w\" (UID: \"d433be25-44af-4432-b487-c1be830fb4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-dtg6w" Jun 15 03:15:20.700258 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.700185 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-dtg6w" Jun 15 03:15:20.831849 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:20.831821 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-dtg6w"] Jun 15 03:15:20.833291 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:15:20.833264 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd433be25_44af_4432_b487_c1be830fb4b8.slice/crio-32747ecb4b91ba5d7154eafae728f667f761027004439f4eaa67218eb9aa434e WatchSource:0}: Error finding container 32747ecb4b91ba5d7154eafae728f667f761027004439f4eaa67218eb9aa434e: Status 404 returned error can't find the container with id 32747ecb4b91ba5d7154eafae728f667f761027004439f4eaa67218eb9aa434e Jun 15 03:15:21.685422 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:21.685385 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-dtg6w" event={"ID":"d433be25-44af-4432-b487-c1be830fb4b8","Type":"ContainerStarted","Data":"32747ecb4b91ba5d7154eafae728f667f761027004439f4eaa67218eb9aa434e"} Jun 15 03:15:22.691634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:22.691600 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-dtg6w" event={"ID":"d433be25-44af-4432-b487-c1be830fb4b8","Type":"ContainerStarted","Data":"444335f674b385b1c2d8df6eff09d9f19805909d9fc8f7f3b8124f02890f99ce"} Jun 15 03:15:22.715943 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:22.715885 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-dtg6w" podStartSLOduration=0.993190144 podStartE2EDuration="2.715869593s" totalImagesPullingTime="1.722679449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:15:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:15:22.713326462 +0000 UTC m=+400.869940670" watchObservedRunningTime="2026-06-15 03:15:22.715869593 +0000 UTC m=+400.872483801" Jun 15 03:15:24.102059 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.102020 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd"] Jun 15 03:15:24.104366 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.104350 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" Jun 15 03:15:24.107265 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.107245 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 03:15:24.107386 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.107307 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 03:15:24.108535 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.108515 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9h8zm\"" Jun 15 03:15:24.114738 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.114719 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd"] Jun 15 03:15:24.213797 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.213766 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b16511f4-89df-42a9-bdc7-c1136b2b6df4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd\" (UID: \"b16511f4-89df-42a9-bdc7-c1136b2b6df4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" Jun 15 03:15:24.214001 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.213816 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b16511f4-89df-42a9-bdc7-c1136b2b6df4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd\" (UID: \"b16511f4-89df-42a9-bdc7-c1136b2b6df4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" Jun 15 03:15:24.214001 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.213896 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rdv4\" (UniqueName: \"kubernetes.io/projected/b16511f4-89df-42a9-bdc7-c1136b2b6df4-kube-api-access-4rdv4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd\" (UID: \"b16511f4-89df-42a9-bdc7-c1136b2b6df4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" Jun 15 03:15:24.315044 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.315009 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b16511f4-89df-42a9-bdc7-c1136b2b6df4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd\" (UID: \"b16511f4-89df-42a9-bdc7-c1136b2b6df4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" Jun 15 03:15:24.315218 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.315100 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b16511f4-89df-42a9-bdc7-c1136b2b6df4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd\" (UID: \"b16511f4-89df-42a9-bdc7-c1136b2b6df4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" Jun 15 03:15:24.315218 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.315172 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rdv4\" (UniqueName: \"kubernetes.io/projected/b16511f4-89df-42a9-bdc7-c1136b2b6df4-kube-api-access-4rdv4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd\" (UID: \"b16511f4-89df-42a9-bdc7-c1136b2b6df4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" Jun 15 03:15:24.315406 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.315387 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b16511f4-89df-42a9-bdc7-c1136b2b6df4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd\" (UID: \"b16511f4-89df-42a9-bdc7-c1136b2b6df4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" Jun 15 03:15:24.315476 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.315456 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b16511f4-89df-42a9-bdc7-c1136b2b6df4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd\" (UID: \"b16511f4-89df-42a9-bdc7-c1136b2b6df4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" Jun 15 03:15:24.323737 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.323713 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rdv4\" (UniqueName: \"kubernetes.io/projected/b16511f4-89df-42a9-bdc7-c1136b2b6df4-kube-api-access-4rdv4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd\" (UID: \"b16511f4-89df-42a9-bdc7-c1136b2b6df4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" Jun 15 03:15:24.413981 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.413899 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" Jun 15 03:15:24.537414 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.537388 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd"] Jun 15 03:15:24.539205 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:15:24.539175 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb16511f4_89df_42a9_bdc7_c1136b2b6df4.slice/crio-09050fb41cda1acbbba5efac587c4e44f03bf026c338159c1888e88b093ca9ee WatchSource:0}: Error finding container 09050fb41cda1acbbba5efac587c4e44f03bf026c338159c1888e88b093ca9ee: Status 404 returned error can't find the container with id 09050fb41cda1acbbba5efac587c4e44f03bf026c338159c1888e88b093ca9ee Jun 15 03:15:24.701014 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.700946 2560 generic.go:356] "Generic (PLEG): container finished" podID="b16511f4-89df-42a9-bdc7-c1136b2b6df4" containerID="57b7a99d1aafc6f2f2a0b9217cb7d4eb5dccd7394cf6612fcf0a2c22625c48c5" exitCode=0 Jun 15 03:15:24.701014 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.700983 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" event={"ID":"b16511f4-89df-42a9-bdc7-c1136b2b6df4","Type":"ContainerDied","Data":"57b7a99d1aafc6f2f2a0b9217cb7d4eb5dccd7394cf6612fcf0a2c22625c48c5"} Jun 15 03:15:24.701194 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:24.701017 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" event={"ID":"b16511f4-89df-42a9-bdc7-c1136b2b6df4","Type":"ContainerStarted","Data":"09050fb41cda1acbbba5efac587c4e44f03bf026c338159c1888e88b093ca9ee"} Jun 15 03:15:27.715810 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:27.715773 2560 generic.go:356] "Generic (PLEG): container finished" podID="b16511f4-89df-42a9-bdc7-c1136b2b6df4" containerID="4bce33088528e9e97d241a8ccae71c1dcd351806e3ee1589f91b148bef0ff53f" exitCode=0 Jun 15 03:15:27.716302 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:27.715861 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" event={"ID":"b16511f4-89df-42a9-bdc7-c1136b2b6df4","Type":"ContainerDied","Data":"4bce33088528e9e97d241a8ccae71c1dcd351806e3ee1589f91b148bef0ff53f"} Jun 15 03:15:28.721739 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:28.721703 2560 generic.go:356] "Generic (PLEG): container finished" podID="b16511f4-89df-42a9-bdc7-c1136b2b6df4" containerID="49e53f090bc7ef4d982d4ea746e515c5a2a0f925b3cc2404c4a6cb3b0d9c0070" exitCode=0 Jun 15 03:15:28.722122 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:28.721775 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" event={"ID":"b16511f4-89df-42a9-bdc7-c1136b2b6df4","Type":"ContainerDied","Data":"49e53f090bc7ef4d982d4ea746e515c5a2a0f925b3cc2404c4a6cb3b0d9c0070"} Jun 15 03:15:29.850322 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:29.850297 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" Jun 15 03:15:29.961440 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:29.961408 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rdv4\" (UniqueName: \"kubernetes.io/projected/b16511f4-89df-42a9-bdc7-c1136b2b6df4-kube-api-access-4rdv4\") pod \"b16511f4-89df-42a9-bdc7-c1136b2b6df4\" (UID: \"b16511f4-89df-42a9-bdc7-c1136b2b6df4\") " Jun 15 03:15:29.961596 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:29.961463 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b16511f4-89df-42a9-bdc7-c1136b2b6df4-util\") pod \"b16511f4-89df-42a9-bdc7-c1136b2b6df4\" (UID: \"b16511f4-89df-42a9-bdc7-c1136b2b6df4\") " Jun 15 03:15:29.961596 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:29.961584 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b16511f4-89df-42a9-bdc7-c1136b2b6df4-bundle\") pod \"b16511f4-89df-42a9-bdc7-c1136b2b6df4\" (UID: \"b16511f4-89df-42a9-bdc7-c1136b2b6df4\") " Jun 15 03:15:29.961925 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:29.961904 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b16511f4-89df-42a9-bdc7-c1136b2b6df4-bundle" (OuterVolumeSpecName: "bundle") pod "b16511f4-89df-42a9-bdc7-c1136b2b6df4" (UID: "b16511f4-89df-42a9-bdc7-c1136b2b6df4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:15:29.963665 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:29.963640 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b16511f4-89df-42a9-bdc7-c1136b2b6df4-kube-api-access-4rdv4" (OuterVolumeSpecName: "kube-api-access-4rdv4") pod "b16511f4-89df-42a9-bdc7-c1136b2b6df4" (UID: "b16511f4-89df-42a9-bdc7-c1136b2b6df4"). InnerVolumeSpecName "kube-api-access-4rdv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:15:29.965954 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:29.965918 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b16511f4-89df-42a9-bdc7-c1136b2b6df4-util" (OuterVolumeSpecName: "util") pod "b16511f4-89df-42a9-bdc7-c1136b2b6df4" (UID: "b16511f4-89df-42a9-bdc7-c1136b2b6df4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:15:30.062633 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:30.062598 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b16511f4-89df-42a9-bdc7-c1136b2b6df4-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:15:30.062633 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:30.062631 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rdv4\" (UniqueName: \"kubernetes.io/projected/b16511f4-89df-42a9-bdc7-c1136b2b6df4-kube-api-access-4rdv4\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:15:30.062818 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:30.062646 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b16511f4-89df-42a9-bdc7-c1136b2b6df4-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:15:30.732416 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:30.732380 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" event={"ID":"b16511f4-89df-42a9-bdc7-c1136b2b6df4","Type":"ContainerDied","Data":"09050fb41cda1acbbba5efac587c4e44f03bf026c338159c1888e88b093ca9ee"} Jun 15 03:15:30.732416 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:30.732410 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrlfd" Jun 15 03:15:30.732616 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:30.732432 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09050fb41cda1acbbba5efac587c4e44f03bf026c338159c1888e88b093ca9ee" Jun 15 03:15:31.418887 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.418852 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-gvbdb"] Jun 15 03:15:31.419275 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.419188 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b16511f4-89df-42a9-bdc7-c1136b2b6df4" containerName="pull" Jun 15 03:15:31.419275 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.419200 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b16511f4-89df-42a9-bdc7-c1136b2b6df4" containerName="pull" Jun 15 03:15:31.419275 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.419209 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b16511f4-89df-42a9-bdc7-c1136b2b6df4" containerName="extract" Jun 15 03:15:31.419275 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.419216 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b16511f4-89df-42a9-bdc7-c1136b2b6df4" containerName="extract" Jun 15 03:15:31.419275 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.419234 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b16511f4-89df-42a9-bdc7-c1136b2b6df4" containerName="util" Jun 15 03:15:31.419275 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.419239 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b16511f4-89df-42a9-bdc7-c1136b2b6df4" containerName="util" Jun 15 03:15:31.419452 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.419284 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b16511f4-89df-42a9-bdc7-c1136b2b6df4" containerName="extract" Jun 15 03:15:31.421174 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.421155 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-gvbdb" Jun 15 03:15:31.424351 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.424331 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 15 03:15:31.425661 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.425640 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 15 03:15:31.425748 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.425692 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-l5qwp\"" Jun 15 03:15:31.430032 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.430012 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-gvbdb"] Jun 15 03:15:31.575128 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.575073 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12c6dfbd-eb64-45ec-b4a9-823854228cee-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-gvbdb\" (UID: \"12c6dfbd-eb64-45ec-b4a9-823854228cee\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-gvbdb" Jun 15 03:15:31.575276 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.575155 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lwjg\" (UniqueName: \"kubernetes.io/projected/12c6dfbd-eb64-45ec-b4a9-823854228cee-kube-api-access-9lwjg\") pod \"cert-manager-cainjector-5d99449ffd-gvbdb\" (UID: \"12c6dfbd-eb64-45ec-b4a9-823854228cee\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-gvbdb" Jun 15 03:15:31.675580 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.675504 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12c6dfbd-eb64-45ec-b4a9-823854228cee-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-gvbdb\" (UID: \"12c6dfbd-eb64-45ec-b4a9-823854228cee\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-gvbdb" Jun 15 03:15:31.675580 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.675548 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lwjg\" (UniqueName: \"kubernetes.io/projected/12c6dfbd-eb64-45ec-b4a9-823854228cee-kube-api-access-9lwjg\") pod \"cert-manager-cainjector-5d99449ffd-gvbdb\" (UID: \"12c6dfbd-eb64-45ec-b4a9-823854228cee\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-gvbdb" Jun 15 03:15:31.683509 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.683485 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12c6dfbd-eb64-45ec-b4a9-823854228cee-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-gvbdb\" (UID: \"12c6dfbd-eb64-45ec-b4a9-823854228cee\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-gvbdb" Jun 15 03:15:31.683625 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.683534 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lwjg\" (UniqueName: \"kubernetes.io/projected/12c6dfbd-eb64-45ec-b4a9-823854228cee-kube-api-access-9lwjg\") pod \"cert-manager-cainjector-5d99449ffd-gvbdb\" (UID: \"12c6dfbd-eb64-45ec-b4a9-823854228cee\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-gvbdb" Jun 15 03:15:31.742704 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.742680 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-gvbdb" Jun 15 03:15:31.866281 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:31.866116 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-gvbdb"] Jun 15 03:15:31.868445 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:15:31.868420 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12c6dfbd_eb64_45ec_b4a9_823854228cee.slice/crio-a9408b599d56ac7d876afd96716d432c12d5d5a782a2d53ff0aa05464d684458 WatchSource:0}: Error finding container a9408b599d56ac7d876afd96716d432c12d5d5a782a2d53ff0aa05464d684458: Status 404 returned error can't find the container with id a9408b599d56ac7d876afd96716d432c12d5d5a782a2d53ff0aa05464d684458 Jun 15 03:15:32.745129 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:32.745077 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-gvbdb" event={"ID":"12c6dfbd-eb64-45ec-b4a9-823854228cee","Type":"ContainerStarted","Data":"a9408b599d56ac7d876afd96716d432c12d5d5a782a2d53ff0aa05464d684458"} Jun 15 03:15:34.757597 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:34.757557 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-gvbdb" event={"ID":"12c6dfbd-eb64-45ec-b4a9-823854228cee","Type":"ContainerStarted","Data":"a52408ea7104cde3be49a003373c85516ac138308a7e6141136f4261b1c39e77"} Jun 15 03:15:34.771704 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:34.771646 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5d99449ffd-gvbdb" podStartSLOduration=1.370541604 podStartE2EDuration="3.771627368s" totalImagesPullingTime="2.401085764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:15:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:15:34.771476948 +0000 UTC m=+412.928091156" watchObservedRunningTime="2026-06-15 03:15:34.771627368 +0000 UTC m=+412.928241578" Jun 15 03:15:43.932185 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:43.932143 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-6gg2g"] Jun 15 03:15:43.934350 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:43.934333 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-6gg2g" Jun 15 03:15:43.936827 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:43.936809 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-nkcbj\"" Jun 15 03:15:43.943313 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:43.943290 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-6gg2g"] Jun 15 03:15:44.077716 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.077686 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj2mc\" (UniqueName: \"kubernetes.io/projected/63db52cd-0430-4819-a3aa-c9eb3a3f83d0-kube-api-access-bj2mc\") pod \"cert-manager-59c77684cc-6gg2g\" (UID: \"63db52cd-0430-4819-a3aa-c9eb3a3f83d0\") " pod="cert-manager/cert-manager-59c77684cc-6gg2g" Jun 15 03:15:44.077716 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.077722 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63db52cd-0430-4819-a3aa-c9eb3a3f83d0-bound-sa-token\") pod \"cert-manager-59c77684cc-6gg2g\" (UID: \"63db52cd-0430-4819-a3aa-c9eb3a3f83d0\") " pod="cert-manager/cert-manager-59c77684cc-6gg2g" Jun 15 03:15:44.179217 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.179178 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj2mc\" (UniqueName: \"kubernetes.io/projected/63db52cd-0430-4819-a3aa-c9eb3a3f83d0-kube-api-access-bj2mc\") pod \"cert-manager-59c77684cc-6gg2g\" (UID: \"63db52cd-0430-4819-a3aa-c9eb3a3f83d0\") " pod="cert-manager/cert-manager-59c77684cc-6gg2g" Jun 15 03:15:44.179217 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.179221 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63db52cd-0430-4819-a3aa-c9eb3a3f83d0-bound-sa-token\") pod \"cert-manager-59c77684cc-6gg2g\" (UID: \"63db52cd-0430-4819-a3aa-c9eb3a3f83d0\") " pod="cert-manager/cert-manager-59c77684cc-6gg2g" Jun 15 03:15:44.187582 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.187523 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63db52cd-0430-4819-a3aa-c9eb3a3f83d0-bound-sa-token\") pod \"cert-manager-59c77684cc-6gg2g\" (UID: \"63db52cd-0430-4819-a3aa-c9eb3a3f83d0\") " pod="cert-manager/cert-manager-59c77684cc-6gg2g" Jun 15 03:15:44.187687 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.187599 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj2mc\" (UniqueName: \"kubernetes.io/projected/63db52cd-0430-4819-a3aa-c9eb3a3f83d0-kube-api-access-bj2mc\") pod \"cert-manager-59c77684cc-6gg2g\" (UID: \"63db52cd-0430-4819-a3aa-c9eb3a3f83d0\") " pod="cert-manager/cert-manager-59c77684cc-6gg2g" Jun 15 03:15:44.243488 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.243461 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-6gg2g" Jun 15 03:15:44.368485 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.368450 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-6gg2g"] Jun 15 03:15:44.373612 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:15:44.373584 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63db52cd_0430_4819_a3aa_c9eb3a3f83d0.slice/crio-de860920a0c773ab6dab02c9fa1b9fb6166a7d08642d6928abc4eefe73cfd6c1 WatchSource:0}: Error finding container de860920a0c773ab6dab02c9fa1b9fb6166a7d08642d6928abc4eefe73cfd6c1: Status 404 returned error can't find the container with id de860920a0c773ab6dab02c9fa1b9fb6166a7d08642d6928abc4eefe73cfd6c1 Jun 15 03:15:44.801288 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.801253 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-6gg2g" event={"ID":"63db52cd-0430-4819-a3aa-c9eb3a3f83d0","Type":"ContainerStarted","Data":"3d11b17f667b0dee029082706b4309d2afe1f32b6592b2ba9a5f946f90696924"} Jun 15 03:15:44.801288 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.801291 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-6gg2g" event={"ID":"63db52cd-0430-4819-a3aa-c9eb3a3f83d0","Type":"ContainerStarted","Data":"de860920a0c773ab6dab02c9fa1b9fb6166a7d08642d6928abc4eefe73cfd6c1"} Jun 15 03:15:44.817501 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.817444 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-6gg2g" podStartSLOduration=1.817428445 podStartE2EDuration="1.817428445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:15:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:15:44.816191735 +0000 UTC m=+422.972805968" watchObservedRunningTime="2026-06-15 03:15:44.817428445 +0000 UTC m=+422.974042630" Jun 15 03:15:44.920057 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.920020 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8"] Jun 15 03:15:44.922249 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.922234 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" Jun 15 03:15:44.924888 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.924858 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 03:15:44.925021 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.924904 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9h8zm\"" Jun 15 03:15:44.925021 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.924904 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 03:15:44.931050 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:44.931028 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8"] Jun 15 03:15:45.088240 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:45.088154 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73487624-fb09-4bdb-8173-8a10c2b18504-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8\" (UID: \"73487624-fb09-4bdb-8173-8a10c2b18504\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" Jun 15 03:15:45.088620 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:45.088236 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqd4g\" (UniqueName: \"kubernetes.io/projected/73487624-fb09-4bdb-8173-8a10c2b18504-kube-api-access-bqd4g\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8\" (UID: \"73487624-fb09-4bdb-8173-8a10c2b18504\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" Jun 15 03:15:45.088620 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:45.088296 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73487624-fb09-4bdb-8173-8a10c2b18504-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8\" (UID: \"73487624-fb09-4bdb-8173-8a10c2b18504\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" Jun 15 03:15:45.188691 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:45.188654 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqd4g\" (UniqueName: \"kubernetes.io/projected/73487624-fb09-4bdb-8173-8a10c2b18504-kube-api-access-bqd4g\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8\" (UID: \"73487624-fb09-4bdb-8173-8a10c2b18504\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" Jun 15 03:15:45.188852 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:45.188723 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73487624-fb09-4bdb-8173-8a10c2b18504-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8\" (UID: \"73487624-fb09-4bdb-8173-8a10c2b18504\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" Jun 15 03:15:45.188852 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:45.188774 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73487624-fb09-4bdb-8173-8a10c2b18504-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8\" (UID: \"73487624-fb09-4bdb-8173-8a10c2b18504\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" Jun 15 03:15:45.189135 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:45.189074 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73487624-fb09-4bdb-8173-8a10c2b18504-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8\" (UID: \"73487624-fb09-4bdb-8173-8a10c2b18504\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" Jun 15 03:15:45.189273 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:45.189144 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73487624-fb09-4bdb-8173-8a10c2b18504-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8\" (UID: \"73487624-fb09-4bdb-8173-8a10c2b18504\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" Jun 15 03:15:45.199855 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:45.199832 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqd4g\" (UniqueName: \"kubernetes.io/projected/73487624-fb09-4bdb-8173-8a10c2b18504-kube-api-access-bqd4g\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8\" (UID: \"73487624-fb09-4bdb-8173-8a10c2b18504\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" Jun 15 03:15:45.232677 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:45.232652 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" Jun 15 03:15:45.359210 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:45.359183 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8"] Jun 15 03:15:45.360686 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:15:45.360657 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73487624_fb09_4bdb_8173_8a10c2b18504.slice/crio-48bc053bc6e2ebe1c23420f7f864b7e98573e6518d3a98cab4891d326fcecca8 WatchSource:0}: Error finding container 48bc053bc6e2ebe1c23420f7f864b7e98573e6518d3a98cab4891d326fcecca8: Status 404 returned error can't find the container with id 48bc053bc6e2ebe1c23420f7f864b7e98573e6518d3a98cab4891d326fcecca8 Jun 15 03:15:45.806996 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:45.806962 2560 generic.go:356] "Generic (PLEG): container finished" podID="73487624-fb09-4bdb-8173-8a10c2b18504" containerID="112b8fe14e083365c26fc06f677aefba578c97e12c77d9210b6de8128e74c675" exitCode=0 Jun 15 03:15:45.807187 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:45.807031 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" event={"ID":"73487624-fb09-4bdb-8173-8a10c2b18504","Type":"ContainerDied","Data":"112b8fe14e083365c26fc06f677aefba578c97e12c77d9210b6de8128e74c675"} Jun 15 03:15:45.807187 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:45.807068 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" event={"ID":"73487624-fb09-4bdb-8173-8a10c2b18504","Type":"ContainerStarted","Data":"48bc053bc6e2ebe1c23420f7f864b7e98573e6518d3a98cab4891d326fcecca8"} Jun 15 03:15:47.817814 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:47.817781 2560 generic.go:356] "Generic (PLEG): container finished" podID="73487624-fb09-4bdb-8173-8a10c2b18504" containerID="580faaca2785836f7345edcb7783a25c65c247b1b76cafb4043c8a40ad25acbf" exitCode=0 Jun 15 03:15:47.818222 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:47.817835 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" event={"ID":"73487624-fb09-4bdb-8173-8a10c2b18504","Type":"ContainerDied","Data":"580faaca2785836f7345edcb7783a25c65c247b1b76cafb4043c8a40ad25acbf"} Jun 15 03:15:48.823856 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:48.823817 2560 generic.go:356] "Generic (PLEG): container finished" podID="73487624-fb09-4bdb-8173-8a10c2b18504" containerID="0dc8b13b920e76ae60ef9efbcff013a3c8d7037c9e7d9dd03032e932613fc8ff" exitCode=0 Jun 15 03:15:48.824248 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:48.823904 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" event={"ID":"73487624-fb09-4bdb-8173-8a10c2b18504","Type":"ContainerDied","Data":"0dc8b13b920e76ae60ef9efbcff013a3c8d7037c9e7d9dd03032e932613fc8ff"} Jun 15 03:15:49.961513 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:49.961493 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" Jun 15 03:15:50.129819 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:50.129734 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqd4g\" (UniqueName: \"kubernetes.io/projected/73487624-fb09-4bdb-8173-8a10c2b18504-kube-api-access-bqd4g\") pod \"73487624-fb09-4bdb-8173-8a10c2b18504\" (UID: \"73487624-fb09-4bdb-8173-8a10c2b18504\") " Jun 15 03:15:50.129819 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:50.129781 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73487624-fb09-4bdb-8173-8a10c2b18504-bundle\") pod \"73487624-fb09-4bdb-8173-8a10c2b18504\" (UID: \"73487624-fb09-4bdb-8173-8a10c2b18504\") " Jun 15 03:15:50.130000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:50.129824 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73487624-fb09-4bdb-8173-8a10c2b18504-util\") pod \"73487624-fb09-4bdb-8173-8a10c2b18504\" (UID: \"73487624-fb09-4bdb-8173-8a10c2b18504\") " Jun 15 03:15:50.130530 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:50.130497 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73487624-fb09-4bdb-8173-8a10c2b18504-bundle" (OuterVolumeSpecName: "bundle") pod "73487624-fb09-4bdb-8173-8a10c2b18504" (UID: "73487624-fb09-4bdb-8173-8a10c2b18504"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:15:50.131866 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:50.131844 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73487624-fb09-4bdb-8173-8a10c2b18504-kube-api-access-bqd4g" (OuterVolumeSpecName: "kube-api-access-bqd4g") pod "73487624-fb09-4bdb-8173-8a10c2b18504" (UID: "73487624-fb09-4bdb-8173-8a10c2b18504"). InnerVolumeSpecName "kube-api-access-bqd4g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:15:50.135073 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:50.135053 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73487624-fb09-4bdb-8173-8a10c2b18504-util" (OuterVolumeSpecName: "util") pod "73487624-fb09-4bdb-8173-8a10c2b18504" (UID: "73487624-fb09-4bdb-8173-8a10c2b18504"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:15:50.231029 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:50.230995 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73487624-fb09-4bdb-8173-8a10c2b18504-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:15:50.231029 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:50.231021 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73487624-fb09-4bdb-8173-8a10c2b18504-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:15:50.231029 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:50.231030 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqd4g\" (UniqueName: \"kubernetes.io/projected/73487624-fb09-4bdb-8173-8a10c2b18504-kube-api-access-bqd4g\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:15:50.837099 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:50.837059 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" Jun 15 03:15:50.837270 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:50.837100 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdzpgv8" event={"ID":"73487624-fb09-4bdb-8173-8a10c2b18504","Type":"ContainerDied","Data":"48bc053bc6e2ebe1c23420f7f864b7e98573e6518d3a98cab4891d326fcecca8"} Jun 15 03:15:50.837270 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:50.837151 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48bc053bc6e2ebe1c23420f7f864b7e98573e6518d3a98cab4891d326fcecca8" Jun 15 03:15:56.859455 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.859369 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7"] Jun 15 03:15:56.859893 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.859876 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73487624-fb09-4bdb-8173-8a10c2b18504" containerName="pull" Jun 15 03:15:56.859951 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.859898 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="73487624-fb09-4bdb-8173-8a10c2b18504" containerName="pull" Jun 15 03:15:56.859951 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.859928 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73487624-fb09-4bdb-8173-8a10c2b18504" containerName="util" Jun 15 03:15:56.859951 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.859937 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="73487624-fb09-4bdb-8173-8a10c2b18504" containerName="util" Jun 15 03:15:56.860058 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.859954 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73487624-fb09-4bdb-8173-8a10c2b18504" containerName="extract" Jun 15 03:15:56.860058 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.859963 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="73487624-fb09-4bdb-8173-8a10c2b18504" containerName="extract" Jun 15 03:15:56.860058 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.860041 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="73487624-fb09-4bdb-8173-8a10c2b18504" containerName="extract" Jun 15 03:15:56.862422 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.862404 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:56.865854 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.865834 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tzlp8\"" Jun 15 03:15:56.865977 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.865858 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 15 03:15:56.866109 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.866075 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 15 03:15:56.866109 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.866099 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 15 03:15:56.866255 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.866075 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 15 03:15:56.886852 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.886830 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c12ee043-c8d5-4ad7-b955-bb125df4d5a0-tmp\") pod \"opendatahub-operator-controller-manager-5799f64c8f-zd8s7\" (UID: \"c12ee043-c8d5-4ad7-b955-bb125df4d5a0\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:56.886986 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.886859 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwdfl\" (UniqueName: \"kubernetes.io/projected/c12ee043-c8d5-4ad7-b955-bb125df4d5a0-kube-api-access-kwdfl\") pod \"opendatahub-operator-controller-manager-5799f64c8f-zd8s7\" (UID: \"c12ee043-c8d5-4ad7-b955-bb125df4d5a0\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:56.886986 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.886891 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c12ee043-c8d5-4ad7-b955-bb125df4d5a0-webhook-cert\") pod \"opendatahub-operator-controller-manager-5799f64c8f-zd8s7\" (UID: \"c12ee043-c8d5-4ad7-b955-bb125df4d5a0\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:56.886986 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.886924 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c12ee043-c8d5-4ad7-b955-bb125df4d5a0-manifests\") pod \"opendatahub-operator-controller-manager-5799f64c8f-zd8s7\" (UID: \"c12ee043-c8d5-4ad7-b955-bb125df4d5a0\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:56.886986 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.886946 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c12ee043-c8d5-4ad7-b955-bb125df4d5a0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5799f64c8f-zd8s7\" (UID: \"c12ee043-c8d5-4ad7-b955-bb125df4d5a0\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:56.890248 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.890223 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7"] Jun 15 03:15:56.987573 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.987531 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c12ee043-c8d5-4ad7-b955-bb125df4d5a0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5799f64c8f-zd8s7\" (UID: \"c12ee043-c8d5-4ad7-b955-bb125df4d5a0\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:56.987768 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.987639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c12ee043-c8d5-4ad7-b955-bb125df4d5a0-tmp\") pod \"opendatahub-operator-controller-manager-5799f64c8f-zd8s7\" (UID: \"c12ee043-c8d5-4ad7-b955-bb125df4d5a0\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:56.987768 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.987669 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwdfl\" (UniqueName: \"kubernetes.io/projected/c12ee043-c8d5-4ad7-b955-bb125df4d5a0-kube-api-access-kwdfl\") pod \"opendatahub-operator-controller-manager-5799f64c8f-zd8s7\" (UID: \"c12ee043-c8d5-4ad7-b955-bb125df4d5a0\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:56.987768 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.987703 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c12ee043-c8d5-4ad7-b955-bb125df4d5a0-webhook-cert\") pod \"opendatahub-operator-controller-manager-5799f64c8f-zd8s7\" (UID: \"c12ee043-c8d5-4ad7-b955-bb125df4d5a0\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:56.987768 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.987754 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c12ee043-c8d5-4ad7-b955-bb125df4d5a0-manifests\") pod \"opendatahub-operator-controller-manager-5799f64c8f-zd8s7\" (UID: \"c12ee043-c8d5-4ad7-b955-bb125df4d5a0\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:56.988177 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.988148 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c12ee043-c8d5-4ad7-b955-bb125df4d5a0-manifests\") pod \"opendatahub-operator-controller-manager-5799f64c8f-zd8s7\" (UID: \"c12ee043-c8d5-4ad7-b955-bb125df4d5a0\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:56.989905 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.989883 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c12ee043-c8d5-4ad7-b955-bb125df4d5a0-tmp\") pod \"opendatahub-operator-controller-manager-5799f64c8f-zd8s7\" (UID: \"c12ee043-c8d5-4ad7-b955-bb125df4d5a0\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:56.990034 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.990015 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c12ee043-c8d5-4ad7-b955-bb125df4d5a0-webhook-cert\") pod \"opendatahub-operator-controller-manager-5799f64c8f-zd8s7\" (UID: \"c12ee043-c8d5-4ad7-b955-bb125df4d5a0\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:56.990115 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.990075 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c12ee043-c8d5-4ad7-b955-bb125df4d5a0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5799f64c8f-zd8s7\" (UID: \"c12ee043-c8d5-4ad7-b955-bb125df4d5a0\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:56.995891 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:56.995870 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwdfl\" (UniqueName: \"kubernetes.io/projected/c12ee043-c8d5-4ad7-b955-bb125df4d5a0-kube-api-access-kwdfl\") pod \"opendatahub-operator-controller-manager-5799f64c8f-zd8s7\" (UID: \"c12ee043-c8d5-4ad7-b955-bb125df4d5a0\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:57.173139 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:57.173020 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:15:57.306121 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:57.306078 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7"] Jun 15 03:15:57.308735 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:15:57.308708 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc12ee043_c8d5_4ad7_b955_bb125df4d5a0.slice/crio-7d0ce10b32434455efae13bd5123d2cc8f9f4144e35bbff9b9af8c0628ad927d WatchSource:0}: Error finding container 7d0ce10b32434455efae13bd5123d2cc8f9f4144e35bbff9b9af8c0628ad927d: Status 404 returned error can't find the container with id 7d0ce10b32434455efae13bd5123d2cc8f9f4144e35bbff9b9af8c0628ad927d Jun 15 03:15:57.868494 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:15:57.868456 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" event={"ID":"c12ee043-c8d5-4ad7-b955-bb125df4d5a0","Type":"ContainerStarted","Data":"7d0ce10b32434455efae13bd5123d2cc8f9f4144e35bbff9b9af8c0628ad927d"} Jun 15 03:16:00.884787 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:00.884754 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" event={"ID":"c12ee043-c8d5-4ad7-b955-bb125df4d5a0","Type":"ContainerStarted","Data":"2722bce4305a22cc4a46cacad276ebf197f74cc39803e69cc4254878df7a1a4b"} Jun 15 03:16:01.890697 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:01.890661 2560 generic.go:356] "Generic (PLEG): container finished" podID="c12ee043-c8d5-4ad7-b955-bb125df4d5a0" containerID="2722bce4305a22cc4a46cacad276ebf197f74cc39803e69cc4254878df7a1a4b" exitCode=0 Jun 15 03:16:01.891209 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:01.890731 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" event={"ID":"c12ee043-c8d5-4ad7-b955-bb125df4d5a0","Type":"ContainerDied","Data":"2722bce4305a22cc4a46cacad276ebf197f74cc39803e69cc4254878df7a1a4b"} Jun 15 03:16:02.897581 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:02.897546 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" event={"ID":"c12ee043-c8d5-4ad7-b955-bb125df4d5a0","Type":"ContainerStarted","Data":"aa313435be3014f8da292970051b6d18824d2f4fa30150afe6fc3f8cfc393f60"} Jun 15 03:16:02.897963 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:02.897668 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:16:02.923100 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:02.923039 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" podStartSLOduration=2.713190989 podStartE2EDuration="6.92302322s" totalImagesPullingTime="3.210644465s" totalInitContainerRuntime="999.187766ms" isStatefulPod=false podCreationTimestamp="2026-06-15 03:15:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:16:02.919777352 +0000 UTC m=+441.076391573" watchObservedRunningTime="2026-06-15 03:16:02.92302322 +0000 UTC m=+441.079637429" Jun 15 03:16:13.904741 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:13.904703 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-zd8s7" Jun 15 03:16:16.285114 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.285067 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-jnphp"] Jun 15 03:16:16.293208 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.293185 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-jnphp" Jun 15 03:16:16.296165 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.296146 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2l975\"" Jun 15 03:16:16.296265 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.296208 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 15 03:16:16.300798 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.300778 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-jnphp"] Jun 15 03:16:16.347280 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.347254 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq7xb\" (UniqueName: \"kubernetes.io/projected/419a1975-d79b-4623-853b-bba16bc0e091-kube-api-access-rq7xb\") pod \"odh-model-controller-6dc94f689f-jnphp\" (UID: \"419a1975-d79b-4623-853b-bba16bc0e091\") " pod="opendatahub/odh-model-controller-6dc94f689f-jnphp" Jun 15 03:16:16.347428 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.347300 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/419a1975-d79b-4623-853b-bba16bc0e091-cert\") pod \"odh-model-controller-6dc94f689f-jnphp\" (UID: \"419a1975-d79b-4623-853b-bba16bc0e091\") " pod="opendatahub/odh-model-controller-6dc94f689f-jnphp" Jun 15 03:16:16.448306 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.448265 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq7xb\" (UniqueName: \"kubernetes.io/projected/419a1975-d79b-4623-853b-bba16bc0e091-kube-api-access-rq7xb\") pod \"odh-model-controller-6dc94f689f-jnphp\" (UID: \"419a1975-d79b-4623-853b-bba16bc0e091\") " pod="opendatahub/odh-model-controller-6dc94f689f-jnphp" Jun 15 03:16:16.448306 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.448310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/419a1975-d79b-4623-853b-bba16bc0e091-cert\") pod \"odh-model-controller-6dc94f689f-jnphp\" (UID: \"419a1975-d79b-4623-853b-bba16bc0e091\") " pod="opendatahub/odh-model-controller-6dc94f689f-jnphp" Jun 15 03:16:16.448527 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:16:16.448461 2560 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 15 03:16:16.448527 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:16:16.448524 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/419a1975-d79b-4623-853b-bba16bc0e091-cert podName:419a1975-d79b-4623-853b-bba16bc0e091 nodeName:}" failed. No retries permitted until 2026-06-15 03:16:16.948507823 +0000 UTC m=+455.105122010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/419a1975-d79b-4623-853b-bba16bc0e091-cert") pod "odh-model-controller-6dc94f689f-jnphp" (UID: "419a1975-d79b-4623-853b-bba16bc0e091") : secret "odh-model-controller-webhook-cert" not found Jun 15 03:16:16.459138 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.459105 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq7xb\" (UniqueName: \"kubernetes.io/projected/419a1975-d79b-4623-853b-bba16bc0e091-kube-api-access-rq7xb\") pod \"odh-model-controller-6dc94f689f-jnphp\" (UID: \"419a1975-d79b-4623-853b-bba16bc0e091\") " pod="opendatahub/odh-model-controller-6dc94f689f-jnphp" Jun 15 03:16:16.924556 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.924523 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-xsk7z"] Jun 15 03:16:16.927923 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.927907 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-xsk7z" Jun 15 03:16:16.931667 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.931643 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 15 03:16:16.950823 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.950793 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-xsk7z"] Jun 15 03:16:16.952609 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.952582 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/419a1975-d79b-4623-853b-bba16bc0e091-cert\") pod \"odh-model-controller-6dc94f689f-jnphp\" (UID: \"419a1975-d79b-4623-853b-bba16bc0e091\") " pod="opendatahub/odh-model-controller-6dc94f689f-jnphp" Jun 15 03:16:16.954930 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:16.954907 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/419a1975-d79b-4623-853b-bba16bc0e091-cert\") pod \"odh-model-controller-6dc94f689f-jnphp\" (UID: \"419a1975-d79b-4623-853b-bba16bc0e091\") " pod="opendatahub/odh-model-controller-6dc94f689f-jnphp" Jun 15 03:16:17.053404 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:17.053367 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5570bf43-0431-4cd1-8c78-c02ff1ac57e2-tls-certs\") pod \"model-serving-api-5dd488fc48-xsk7z\" (UID: \"5570bf43-0431-4cd1-8c78-c02ff1ac57e2\") " pod="opendatahub/model-serving-api-5dd488fc48-xsk7z" Jun 15 03:16:17.053404 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:17.053406 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq8dq\" (UniqueName: \"kubernetes.io/projected/5570bf43-0431-4cd1-8c78-c02ff1ac57e2-kube-api-access-jq8dq\") pod \"model-serving-api-5dd488fc48-xsk7z\" (UID: \"5570bf43-0431-4cd1-8c78-c02ff1ac57e2\") " pod="opendatahub/model-serving-api-5dd488fc48-xsk7z" Jun 15 03:16:17.154133 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:17.154072 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5570bf43-0431-4cd1-8c78-c02ff1ac57e2-tls-certs\") pod \"model-serving-api-5dd488fc48-xsk7z\" (UID: \"5570bf43-0431-4cd1-8c78-c02ff1ac57e2\") " pod="opendatahub/model-serving-api-5dd488fc48-xsk7z" Jun 15 03:16:17.154133 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:17.154141 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq8dq\" (UniqueName: \"kubernetes.io/projected/5570bf43-0431-4cd1-8c78-c02ff1ac57e2-kube-api-access-jq8dq\") pod \"model-serving-api-5dd488fc48-xsk7z\" (UID: \"5570bf43-0431-4cd1-8c78-c02ff1ac57e2\") " pod="opendatahub/model-serving-api-5dd488fc48-xsk7z" Jun 15 03:16:17.154349 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:16:17.154229 2560 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jun 15 03:16:17.154349 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:16:17.154295 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5570bf43-0431-4cd1-8c78-c02ff1ac57e2-tls-certs podName:5570bf43-0431-4cd1-8c78-c02ff1ac57e2 nodeName:}" failed. No retries permitted until 2026-06-15 03:16:17.654277771 +0000 UTC m=+455.810891957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/5570bf43-0431-4cd1-8c78-c02ff1ac57e2-tls-certs") pod "model-serving-api-5dd488fc48-xsk7z" (UID: "5570bf43-0431-4cd1-8c78-c02ff1ac57e2") : secret "model-serving-api-tls" not found Jun 15 03:16:17.163502 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:17.163473 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq8dq\" (UniqueName: \"kubernetes.io/projected/5570bf43-0431-4cd1-8c78-c02ff1ac57e2-kube-api-access-jq8dq\") pod \"model-serving-api-5dd488fc48-xsk7z\" (UID: \"5570bf43-0431-4cd1-8c78-c02ff1ac57e2\") " pod="opendatahub/model-serving-api-5dd488fc48-xsk7z" Jun 15 03:16:17.203521 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:17.203440 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-jnphp" Jun 15 03:16:17.330401 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:17.330363 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-jnphp"] Jun 15 03:16:17.333273 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:16:17.333248 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod419a1975_d79b_4623_853b_bba16bc0e091.slice/crio-c252bdabb585235029c4f5256a1aaab90b4ad913cd051d2477f7fa9cd919659a WatchSource:0}: Error finding container c252bdabb585235029c4f5256a1aaab90b4ad913cd051d2477f7fa9cd919659a: Status 404 returned error can't find the container with id c252bdabb585235029c4f5256a1aaab90b4ad913cd051d2477f7fa9cd919659a Jun 15 03:16:17.658171 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:17.658114 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5570bf43-0431-4cd1-8c78-c02ff1ac57e2-tls-certs\") pod \"model-serving-api-5dd488fc48-xsk7z\" (UID: \"5570bf43-0431-4cd1-8c78-c02ff1ac57e2\") " pod="opendatahub/model-serving-api-5dd488fc48-xsk7z" Jun 15 03:16:17.660546 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:17.660522 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5570bf43-0431-4cd1-8c78-c02ff1ac57e2-tls-certs\") pod \"model-serving-api-5dd488fc48-xsk7z\" (UID: \"5570bf43-0431-4cd1-8c78-c02ff1ac57e2\") " pod="opendatahub/model-serving-api-5dd488fc48-xsk7z" Jun 15 03:16:17.837703 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:17.837661 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-xsk7z" Jun 15 03:16:17.970538 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:17.970507 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-xsk7z"] Jun 15 03:16:17.971951 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:16:17.971916 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5570bf43_0431_4cd1_8c78_c02ff1ac57e2.slice/crio-3db0730c1403321f0f677d7baa8db22d86302392fb6c0d7aee88f1c3cfaf6ee1 WatchSource:0}: Error finding container 3db0730c1403321f0f677d7baa8db22d86302392fb6c0d7aee88f1c3cfaf6ee1: Status 404 returned error can't find the container with id 3db0730c1403321f0f677d7baa8db22d86302392fb6c0d7aee88f1c3cfaf6ee1 Jun 15 03:16:17.976031 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:17.975964 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-jnphp" event={"ID":"419a1975-d79b-4623-853b-bba16bc0e091","Type":"ContainerStarted","Data":"c252bdabb585235029c4f5256a1aaab90b4ad913cd051d2477f7fa9cd919659a"} Jun 15 03:16:18.982170 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:18.982131 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-xsk7z" event={"ID":"5570bf43-0431-4cd1-8c78-c02ff1ac57e2","Type":"ContainerStarted","Data":"3db0730c1403321f0f677d7baa8db22d86302392fb6c0d7aee88f1c3cfaf6ee1"} Jun 15 03:16:21.474271 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.474206 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4"] Jun 15 03:16:21.478858 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.478834 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4" Jun 15 03:16:21.481529 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.481504 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 15 03:16:21.481670 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.481597 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 15 03:16:21.481670 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.481639 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mm9rc\"" Jun 15 03:16:21.489344 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.489314 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4"] Jun 15 03:16:21.499938 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.499910 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm"] Jun 15 03:16:21.505216 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.505193 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" Jun 15 03:16:21.508456 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.508162 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 03:16:21.508456 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.508312 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9h8zm\"" Jun 15 03:16:21.508456 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.508172 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 03:16:21.515608 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.515587 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm"] Jun 15 03:16:21.594372 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.594342 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87258\" (UniqueName: \"kubernetes.io/projected/77f6c16f-a5eb-4963-abb0-db8b2ad4db30-kube-api-access-87258\") pod \"kube-auth-proxy-7579ff4b78-jgtz4\" (UID: \"77f6c16f-a5eb-4963-abb0-db8b2ad4db30\") " pod="openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4" Jun 15 03:16:21.594517 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.594401 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77f6c16f-a5eb-4963-abb0-db8b2ad4db30-tmp\") pod \"kube-auth-proxy-7579ff4b78-jgtz4\" (UID: \"77f6c16f-a5eb-4963-abb0-db8b2ad4db30\") " pod="openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4" Jun 15 03:16:21.594517 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.594474 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpnmv\" (UniqueName: \"kubernetes.io/projected/bb50e33b-8e53-4919-a62b-bad577cf53d4-kube-api-access-hpnmv\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm\" (UID: \"bb50e33b-8e53-4919-a62b-bad577cf53d4\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" Jun 15 03:16:21.594630 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.594551 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f6c16f-a5eb-4963-abb0-db8b2ad4db30-tls-certs\") pod \"kube-auth-proxy-7579ff4b78-jgtz4\" (UID: \"77f6c16f-a5eb-4963-abb0-db8b2ad4db30\") " pod="openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4" Jun 15 03:16:21.594630 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.594585 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb50e33b-8e53-4919-a62b-bad577cf53d4-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm\" (UID: \"bb50e33b-8e53-4919-a62b-bad577cf53d4\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" Jun 15 03:16:21.594630 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.594612 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb50e33b-8e53-4919-a62b-bad577cf53d4-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm\" (UID: \"bb50e33b-8e53-4919-a62b-bad577cf53d4\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" Jun 15 03:16:21.695363 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.695330 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77f6c16f-a5eb-4963-abb0-db8b2ad4db30-tmp\") pod \"kube-auth-proxy-7579ff4b78-jgtz4\" (UID: \"77f6c16f-a5eb-4963-abb0-db8b2ad4db30\") " pod="openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4" Jun 15 03:16:21.695545 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.695371 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpnmv\" (UniqueName: \"kubernetes.io/projected/bb50e33b-8e53-4919-a62b-bad577cf53d4-kube-api-access-hpnmv\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm\" (UID: \"bb50e33b-8e53-4919-a62b-bad577cf53d4\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" Jun 15 03:16:21.695545 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.695407 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f6c16f-a5eb-4963-abb0-db8b2ad4db30-tls-certs\") pod \"kube-auth-proxy-7579ff4b78-jgtz4\" (UID: \"77f6c16f-a5eb-4963-abb0-db8b2ad4db30\") " pod="openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4" Jun 15 03:16:21.695545 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.695438 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb50e33b-8e53-4919-a62b-bad577cf53d4-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm\" (UID: \"bb50e33b-8e53-4919-a62b-bad577cf53d4\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" Jun 15 03:16:21.695545 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.695474 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb50e33b-8e53-4919-a62b-bad577cf53d4-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm\" (UID: \"bb50e33b-8e53-4919-a62b-bad577cf53d4\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" Jun 15 03:16:21.695545 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.695507 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87258\" (UniqueName: \"kubernetes.io/projected/77f6c16f-a5eb-4963-abb0-db8b2ad4db30-kube-api-access-87258\") pod \"kube-auth-proxy-7579ff4b78-jgtz4\" (UID: \"77f6c16f-a5eb-4963-abb0-db8b2ad4db30\") " pod="openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4" Jun 15 03:16:21.695915 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.695887 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb50e33b-8e53-4919-a62b-bad577cf53d4-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm\" (UID: \"bb50e33b-8e53-4919-a62b-bad577cf53d4\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" Jun 15 03:16:21.696033 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.695916 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb50e33b-8e53-4919-a62b-bad577cf53d4-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm\" (UID: \"bb50e33b-8e53-4919-a62b-bad577cf53d4\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" Jun 15 03:16:21.697577 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.697559 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77f6c16f-a5eb-4963-abb0-db8b2ad4db30-tmp\") pod \"kube-auth-proxy-7579ff4b78-jgtz4\" (UID: \"77f6c16f-a5eb-4963-abb0-db8b2ad4db30\") " pod="openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4" Jun 15 03:16:21.697697 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.697678 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f6c16f-a5eb-4963-abb0-db8b2ad4db30-tls-certs\") pod \"kube-auth-proxy-7579ff4b78-jgtz4\" (UID: \"77f6c16f-a5eb-4963-abb0-db8b2ad4db30\") " pod="openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4" Jun 15 03:16:21.703530 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.703508 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87258\" (UniqueName: \"kubernetes.io/projected/77f6c16f-a5eb-4963-abb0-db8b2ad4db30-kube-api-access-87258\") pod \"kube-auth-proxy-7579ff4b78-jgtz4\" (UID: \"77f6c16f-a5eb-4963-abb0-db8b2ad4db30\") " pod="openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4" Jun 15 03:16:21.703741 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.703718 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpnmv\" (UniqueName: \"kubernetes.io/projected/bb50e33b-8e53-4919-a62b-bad577cf53d4-kube-api-access-hpnmv\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm\" (UID: \"bb50e33b-8e53-4919-a62b-bad577cf53d4\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" Jun 15 03:16:21.792607 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.792569 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4" Jun 15 03:16:21.816884 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.816855 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" Jun 15 03:16:21.934838 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.934649 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4"] Jun 15 03:16:21.936593 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:16:21.936565 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77f6c16f_a5eb_4963_abb0_db8b2ad4db30.slice/crio-c4ef3518e319d2ef2b5bc8204f2e9d0b63b05ba772df83eecaeb537ecf70582e WatchSource:0}: Error finding container c4ef3518e319d2ef2b5bc8204f2e9d0b63b05ba772df83eecaeb537ecf70582e: Status 404 returned error can't find the container with id c4ef3518e319d2ef2b5bc8204f2e9d0b63b05ba772df83eecaeb537ecf70582e Jun 15 03:16:21.967028 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.967003 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm"] Jun 15 03:16:21.969786 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:16:21.969755 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb50e33b_8e53_4919_a62b_bad577cf53d4.slice/crio-42801606a4fa6a912fe1d7bcfb5c47a122d00da0de1ce7f80f016d01bec3bf9b WatchSource:0}: Error finding container 42801606a4fa6a912fe1d7bcfb5c47a122d00da0de1ce7f80f016d01bec3bf9b: Status 404 returned error can't find the container with id 42801606a4fa6a912fe1d7bcfb5c47a122d00da0de1ce7f80f016d01bec3bf9b Jun 15 03:16:21.999243 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.999213 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-jnphp" event={"ID":"419a1975-d79b-4623-853b-bba16bc0e091","Type":"ContainerStarted","Data":"ef7b91ba186fcf47241c3b8c1c2b763544ca989711619f87eaae23fa95da98cb"} Jun 15 03:16:21.999383 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:21.999346 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-6dc94f689f-jnphp" Jun 15 03:16:22.000472 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:22.000450 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4" event={"ID":"77f6c16f-a5eb-4963-abb0-db8b2ad4db30","Type":"ContainerStarted","Data":"c4ef3518e319d2ef2b5bc8204f2e9d0b63b05ba772df83eecaeb537ecf70582e"} Jun 15 03:16:22.002269 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:22.002237 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-xsk7z" event={"ID":"5570bf43-0431-4cd1-8c78-c02ff1ac57e2","Type":"ContainerStarted","Data":"1ef2db387168e30417abbf0861a49f75edd4d48f1f3d4bf307bfb8428f1f9b1b"} Jun 15 03:16:22.002371 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:22.002326 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-5dd488fc48-xsk7z" Jun 15 03:16:22.003547 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:22.003529 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" event={"ID":"bb50e33b-8e53-4919-a62b-bad577cf53d4","Type":"ContainerStarted","Data":"42801606a4fa6a912fe1d7bcfb5c47a122d00da0de1ce7f80f016d01bec3bf9b"} Jun 15 03:16:22.015641 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:22.015597 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-6dc94f689f-jnphp" podStartSLOduration=1.800780764 podStartE2EDuration="6.015584486s" totalImagesPullingTime="4.214803722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:16:22.014675378 +0000 UTC m=+460.171289591" watchObservedRunningTime="2026-06-15 03:16:22.015584486 +0000 UTC m=+460.172198693" Jun 15 03:16:22.032425 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:22.032388 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-5dd488fc48-xsk7z" podStartSLOduration=2.422872811 podStartE2EDuration="6.032374151s" totalImagesPullingTime="3.60950134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:16:22.030741844 +0000 UTC m=+460.187356058" watchObservedRunningTime="2026-06-15 03:16:22.032374151 +0000 UTC m=+460.188988358" Jun 15 03:16:23.028247 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:23.028203 2560 generic.go:356] "Generic (PLEG): container finished" podID="bb50e33b-8e53-4919-a62b-bad577cf53d4" containerID="a09d74f5af0ec653d729461b8cfc212dcb9e59f322e6ae5ea28b7fec56c7d6b7" exitCode=0 Jun 15 03:16:23.028754 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:23.028661 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" event={"ID":"bb50e33b-8e53-4919-a62b-bad577cf53d4","Type":"ContainerDied","Data":"a09d74f5af0ec653d729461b8cfc212dcb9e59f322e6ae5ea28b7fec56c7d6b7"} Jun 15 03:16:25.040040 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:25.040008 2560 generic.go:356] "Generic (PLEG): container finished" podID="bb50e33b-8e53-4919-a62b-bad577cf53d4" containerID="895d273813cbc4d5eefee10bcb6d4ed3ff8e146fd17f15acd19a721d9100c13a" exitCode=0 Jun 15 03:16:25.040435 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:25.040121 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" event={"ID":"bb50e33b-8e53-4919-a62b-bad577cf53d4","Type":"ContainerDied","Data":"895d273813cbc4d5eefee10bcb6d4ed3ff8e146fd17f15acd19a721d9100c13a"} Jun 15 03:16:25.989000 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:25.988963 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-dbmcd"] Jun 15 03:16:25.992338 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:25.992319 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-dbmcd" Jun 15 03:16:25.997208 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:25.997185 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 15 03:16:25.998450 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:25.998428 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-fdxr9\"" Jun 15 03:16:26.010348 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.010322 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr"] Jun 15 03:16:26.013575 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.013557 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr" Jun 15 03:16:26.020651 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.020630 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 15 03:16:26.020917 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.020892 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-zd8gp\"" Jun 15 03:16:26.022641 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.022620 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-dbmcd"] Jun 15 03:16:26.046655 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.046623 2560 generic.go:356] "Generic (PLEG): container finished" podID="bb50e33b-8e53-4919-a62b-bad577cf53d4" containerID="9272ff80cf8d2e9198dca1148c3baf3b8b8b92c6ac16d3dbd29dd4dc9240c78f" exitCode=0 Jun 15 03:16:26.047020 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.046703 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" event={"ID":"bb50e33b-8e53-4919-a62b-bad577cf53d4","Type":"ContainerDied","Data":"9272ff80cf8d2e9198dca1148c3baf3b8b8b92c6ac16d3dbd29dd4dc9240c78f"} Jun 15 03:16:26.048097 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.048062 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4" event={"ID":"77f6c16f-a5eb-4963-abb0-db8b2ad4db30","Type":"ContainerStarted","Data":"9d5d51362e4d1621ad411a833fb10cf166991f9e32d356088a7f318489f901d2"} Jun 15 03:16:26.053012 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.052987 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr"] Jun 15 03:16:26.123986 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.123934 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7579ff4b78-jgtz4" podStartSLOduration=1.962758806 podStartE2EDuration="5.123917712s" totalImagesPullingTime="3.161158906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:16:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:16:26.123042462 +0000 UTC m=+464.279656690" watchObservedRunningTime="2026-06-15 03:16:26.123917712 +0000 UTC m=+464.280531920" Jun 15 03:16:26.133703 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.133671 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f22b40b-22cd-43b1-892b-f0fc87707156-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-xmklr\" (UID: \"3f22b40b-22cd-43b1-892b-f0fc87707156\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr" Jun 15 03:16:26.133867 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.133744 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d22rx\" (UniqueName: \"kubernetes.io/projected/3f22b40b-22cd-43b1-892b-f0fc87707156-kube-api-access-d22rx\") pod \"llmisvc-controller-manager-864bfd7bc9-xmklr\" (UID: \"3f22b40b-22cd-43b1-892b-f0fc87707156\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr" Jun 15 03:16:26.133867 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.133841 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqdcj\" (UniqueName: \"kubernetes.io/projected/9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc-kube-api-access-sqdcj\") pod \"kserve-controller-manager-69fc866d76-dbmcd\" (UID: \"9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc\") " pod="opendatahub/kserve-controller-manager-69fc866d76-dbmcd" Jun 15 03:16:26.133979 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.133939 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc-cert\") pod \"kserve-controller-manager-69fc866d76-dbmcd\" (UID: \"9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc\") " pod="opendatahub/kserve-controller-manager-69fc866d76-dbmcd" Jun 15 03:16:26.234686 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.234653 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc-cert\") pod \"kserve-controller-manager-69fc866d76-dbmcd\" (UID: \"9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc\") " pod="opendatahub/kserve-controller-manager-69fc866d76-dbmcd" Jun 15 03:16:26.234864 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.234696 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f22b40b-22cd-43b1-892b-f0fc87707156-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-xmklr\" (UID: \"3f22b40b-22cd-43b1-892b-f0fc87707156\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr" Jun 15 03:16:26.234864 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.234715 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d22rx\" (UniqueName: \"kubernetes.io/projected/3f22b40b-22cd-43b1-892b-f0fc87707156-kube-api-access-d22rx\") pod \"llmisvc-controller-manager-864bfd7bc9-xmklr\" (UID: \"3f22b40b-22cd-43b1-892b-f0fc87707156\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr" Jun 15 03:16:26.234864 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.234759 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqdcj\" (UniqueName: \"kubernetes.io/projected/9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc-kube-api-access-sqdcj\") pod \"kserve-controller-manager-69fc866d76-dbmcd\" (UID: \"9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc\") " pod="opendatahub/kserve-controller-manager-69fc866d76-dbmcd" Jun 15 03:16:26.237302 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.237276 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc-cert\") pod \"kserve-controller-manager-69fc866d76-dbmcd\" (UID: \"9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc\") " pod="opendatahub/kserve-controller-manager-69fc866d76-dbmcd" Jun 15 03:16:26.237485 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.237464 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f22b40b-22cd-43b1-892b-f0fc87707156-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-xmklr\" (UID: \"3f22b40b-22cd-43b1-892b-f0fc87707156\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr" Jun 15 03:16:26.243467 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.243418 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d22rx\" (UniqueName: \"kubernetes.io/projected/3f22b40b-22cd-43b1-892b-f0fc87707156-kube-api-access-d22rx\") pod \"llmisvc-controller-manager-864bfd7bc9-xmklr\" (UID: \"3f22b40b-22cd-43b1-892b-f0fc87707156\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr" Jun 15 03:16:26.243593 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.243572 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqdcj\" (UniqueName: \"kubernetes.io/projected/9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc-kube-api-access-sqdcj\") pod \"kserve-controller-manager-69fc866d76-dbmcd\" (UID: \"9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc\") " pod="opendatahub/kserve-controller-manager-69fc866d76-dbmcd" Jun 15 03:16:26.303464 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.303408 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-dbmcd" Jun 15 03:16:26.323616 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.323584 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr" Jun 15 03:16:26.454441 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.454412 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-dbmcd"] Jun 15 03:16:26.456422 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:16:26.456394 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bbcc475_5dd9_4830_bcd9_d3b0bebe58dc.slice/crio-cbe900b881cf0ecc5521a03e05a687096d86fe141f39a341a5849c38cbd48888 WatchSource:0}: Error finding container cbe900b881cf0ecc5521a03e05a687096d86fe141f39a341a5849c38cbd48888: Status 404 returned error can't find the container with id cbe900b881cf0ecc5521a03e05a687096d86fe141f39a341a5849c38cbd48888 Jun 15 03:16:26.479947 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:26.479918 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr"] Jun 15 03:16:26.481507 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:16:26.481483 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3f22b40b_22cd_43b1_892b_f0fc87707156.slice/crio-0ad455587298252e407d2651e62167768055f97e9e56d1df48b23280ed979230 WatchSource:0}: Error finding container 0ad455587298252e407d2651e62167768055f97e9e56d1df48b23280ed979230: Status 404 returned error can't find the container with id 0ad455587298252e407d2651e62167768055f97e9e56d1df48b23280ed979230 Jun 15 03:16:27.054492 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:27.054442 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-dbmcd" event={"ID":"9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc","Type":"ContainerStarted","Data":"cbe900b881cf0ecc5521a03e05a687096d86fe141f39a341a5849c38cbd48888"} Jun 15 03:16:27.056390 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:27.056364 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr" event={"ID":"3f22b40b-22cd-43b1-892b-f0fc87707156","Type":"ContainerStarted","Data":"0ad455587298252e407d2651e62167768055f97e9e56d1df48b23280ed979230"} Jun 15 03:16:27.212394 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:27.212358 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" Jun 15 03:16:27.345780 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:27.345697 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpnmv\" (UniqueName: \"kubernetes.io/projected/bb50e33b-8e53-4919-a62b-bad577cf53d4-kube-api-access-hpnmv\") pod \"bb50e33b-8e53-4919-a62b-bad577cf53d4\" (UID: \"bb50e33b-8e53-4919-a62b-bad577cf53d4\") " Jun 15 03:16:27.345780 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:27.345762 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb50e33b-8e53-4919-a62b-bad577cf53d4-util\") pod \"bb50e33b-8e53-4919-a62b-bad577cf53d4\" (UID: \"bb50e33b-8e53-4919-a62b-bad577cf53d4\") " Jun 15 03:16:27.345997 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:27.345793 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb50e33b-8e53-4919-a62b-bad577cf53d4-bundle\") pod \"bb50e33b-8e53-4919-a62b-bad577cf53d4\" (UID: \"bb50e33b-8e53-4919-a62b-bad577cf53d4\") " Jun 15 03:16:27.347161 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:27.347072 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb50e33b-8e53-4919-a62b-bad577cf53d4-bundle" (OuterVolumeSpecName: "bundle") pod "bb50e33b-8e53-4919-a62b-bad577cf53d4" (UID: "bb50e33b-8e53-4919-a62b-bad577cf53d4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:16:27.348317 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:27.348283 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb50e33b-8e53-4919-a62b-bad577cf53d4-kube-api-access-hpnmv" (OuterVolumeSpecName: "kube-api-access-hpnmv") pod "bb50e33b-8e53-4919-a62b-bad577cf53d4" (UID: "bb50e33b-8e53-4919-a62b-bad577cf53d4"). InnerVolumeSpecName "kube-api-access-hpnmv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:16:27.351762 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:27.351738 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb50e33b-8e53-4919-a62b-bad577cf53d4-util" (OuterVolumeSpecName: "util") pod "bb50e33b-8e53-4919-a62b-bad577cf53d4" (UID: "bb50e33b-8e53-4919-a62b-bad577cf53d4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:16:27.447406 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:27.447371 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpnmv\" (UniqueName: \"kubernetes.io/projected/bb50e33b-8e53-4919-a62b-bad577cf53d4-kube-api-access-hpnmv\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:16:27.447406 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:27.447403 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb50e33b-8e53-4919-a62b-bad577cf53d4-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:16:27.447585 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:27.447417 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb50e33b-8e53-4919-a62b-bad577cf53d4-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:16:28.064959 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:28.064922 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" Jun 15 03:16:28.064959 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:28.064938 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8cwm" event={"ID":"bb50e33b-8e53-4919-a62b-bad577cf53d4","Type":"ContainerDied","Data":"42801606a4fa6a912fe1d7bcfb5c47a122d00da0de1ce7f80f016d01bec3bf9b"} Jun 15 03:16:28.065576 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:28.065003 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="42801606a4fa6a912fe1d7bcfb5c47a122d00da0de1ce7f80f016d01bec3bf9b" Jun 15 03:16:30.076442 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:30.076407 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-dbmcd" event={"ID":"9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc","Type":"ContainerStarted","Data":"cd58461d547176bd4de914fb823ad5672ce4ae5d7ef04d4207e4ec44ab3bff85"} Jun 15 03:16:30.076903 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:30.076492 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-69fc866d76-dbmcd" Jun 15 03:16:30.093963 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:30.093911 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-69fc866d76-dbmcd" podStartSLOduration=2.21589726 podStartE2EDuration="5.093895536s" totalImagesPullingTime="2.877998276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:16:30.093413207 +0000 UTC m=+468.250027452" watchObservedRunningTime="2026-06-15 03:16:30.093895536 +0000 UTC m=+468.250509744" Jun 15 03:16:31.083054 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:31.083014 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr" event={"ID":"3f22b40b-22cd-43b1-892b-f0fc87707156","Type":"ContainerStarted","Data":"af68b9dddde12c4f7cd18650a7941a634872cd54e6e02bc4ce549e704bff07c8"} Jun 15 03:16:31.109415 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:31.109365 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr" podStartSLOduration=2.483619406 podStartE2EDuration="6.109350089s" totalImagesPullingTime="3.625730683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:16:31.107809736 +0000 UTC m=+469.264423942" watchObservedRunningTime="2026-06-15 03:16:31.109350089 +0000 UTC m=+469.265964298" Jun 15 03:16:32.087654 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:32.087619 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr" Jun 15 03:16:33.031639 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:33.031608 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-6dc94f689f-jnphp" Jun 15 03:16:33.033386 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:33.033365 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-5dd488fc48-xsk7z" Jun 15 03:16:35.270613 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.270580 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs"] Jun 15 03:16:35.270988 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.270944 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb50e33b-8e53-4919-a62b-bad577cf53d4" containerName="pull" Jun 15 03:16:35.270988 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.270956 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb50e33b-8e53-4919-a62b-bad577cf53d4" containerName="pull" Jun 15 03:16:35.270988 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.270969 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb50e33b-8e53-4919-a62b-bad577cf53d4" containerName="util" Jun 15 03:16:35.270988 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.270974 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb50e33b-8e53-4919-a62b-bad577cf53d4" containerName="util" Jun 15 03:16:35.270988 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.270980 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb50e33b-8e53-4919-a62b-bad577cf53d4" containerName="extract" Jun 15 03:16:35.270988 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.270987 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb50e33b-8e53-4919-a62b-bad577cf53d4" containerName="extract" Jun 15 03:16:35.271200 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.271036 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb50e33b-8e53-4919-a62b-bad577cf53d4" containerName="extract" Jun 15 03:16:35.276844 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.276825 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" Jun 15 03:16:35.279811 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.279789 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 03:16:35.281016 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.280998 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 03:16:35.281116 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.281006 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9h8zm\"" Jun 15 03:16:35.287548 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.287521 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs"] Jun 15 03:16:35.417128 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.417066 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec2e70c7-0439-43e6-8043-093b956877aa-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs\" (UID: \"ec2e70c7-0439-43e6-8043-093b956877aa\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" Jun 15 03:16:35.417351 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.417330 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec2e70c7-0439-43e6-8043-093b956877aa-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs\" (UID: \"ec2e70c7-0439-43e6-8043-093b956877aa\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" Jun 15 03:16:35.417411 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.417387 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldv9x\" (UniqueName: \"kubernetes.io/projected/ec2e70c7-0439-43e6-8043-093b956877aa-kube-api-access-ldv9x\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs\" (UID: \"ec2e70c7-0439-43e6-8043-093b956877aa\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" Jun 15 03:16:35.518447 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.518409 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec2e70c7-0439-43e6-8043-093b956877aa-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs\" (UID: \"ec2e70c7-0439-43e6-8043-093b956877aa\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" Jun 15 03:16:35.518447 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.518448 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec2e70c7-0439-43e6-8043-093b956877aa-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs\" (UID: \"ec2e70c7-0439-43e6-8043-093b956877aa\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" Jun 15 03:16:35.518700 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.518481 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldv9x\" (UniqueName: \"kubernetes.io/projected/ec2e70c7-0439-43e6-8043-093b956877aa-kube-api-access-ldv9x\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs\" (UID: \"ec2e70c7-0439-43e6-8043-093b956877aa\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" Jun 15 03:16:35.518892 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.518872 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec2e70c7-0439-43e6-8043-093b956877aa-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs\" (UID: \"ec2e70c7-0439-43e6-8043-093b956877aa\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" Jun 15 03:16:35.518963 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.518885 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec2e70c7-0439-43e6-8043-093b956877aa-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs\" (UID: \"ec2e70c7-0439-43e6-8043-093b956877aa\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" Jun 15 03:16:35.528598 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.528536 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldv9x\" (UniqueName: \"kubernetes.io/projected/ec2e70c7-0439-43e6-8043-093b956877aa-kube-api-access-ldv9x\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs\" (UID: \"ec2e70c7-0439-43e6-8043-093b956877aa\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" Jun 15 03:16:35.587796 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.587763 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" Jun 15 03:16:35.727612 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:35.727585 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs"] Jun 15 03:16:35.729878 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:16:35.729848 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec2e70c7_0439_43e6_8043_093b956877aa.slice/crio-21470c7a8488a1239b427490f73c18205fdd0d85f4686a7fa440ccf68317945a WatchSource:0}: Error finding container 21470c7a8488a1239b427490f73c18205fdd0d85f4686a7fa440ccf68317945a: Status 404 returned error can't find the container with id 21470c7a8488a1239b427490f73c18205fdd0d85f4686a7fa440ccf68317945a Jun 15 03:16:36.108750 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:36.108657 2560 generic.go:356] "Generic (PLEG): container finished" podID="ec2e70c7-0439-43e6-8043-093b956877aa" containerID="7aa0a720801ce45f30cee93e50e8b405383ec24e85d28c7cf542663570d0123c" exitCode=0 Jun 15 03:16:36.108919 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:36.108751 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" event={"ID":"ec2e70c7-0439-43e6-8043-093b956877aa","Type":"ContainerDied","Data":"7aa0a720801ce45f30cee93e50e8b405383ec24e85d28c7cf542663570d0123c"} Jun 15 03:16:36.108919 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:36.108810 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" event={"ID":"ec2e70c7-0439-43e6-8043-093b956877aa","Type":"ContainerStarted","Data":"21470c7a8488a1239b427490f73c18205fdd0d85f4686a7fa440ccf68317945a"} Jun 15 03:16:36.712363 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:36.712326 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-84drr"] Jun 15 03:16:36.714830 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:36.714814 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-84drr" Jun 15 03:16:36.717852 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:36.717819 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-x8tsf\"" Jun 15 03:16:36.717852 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:36.717819 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 15 03:16:36.718018 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:36.717986 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 15 03:16:36.728503 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:36.728476 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-84drr"] Jun 15 03:16:36.830766 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:36.830681 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn2b6\" (UniqueName: \"kubernetes.io/projected/fc2d3adb-1049-4977-9ae9-ec9285b4c40c-kube-api-access-mn2b6\") pod \"servicemesh-operator3-57f65f65fb-84drr\" (UID: \"fc2d3adb-1049-4977-9ae9-ec9285b4c40c\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-84drr" Jun 15 03:16:36.830766 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:36.830738 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/fc2d3adb-1049-4977-9ae9-ec9285b4c40c-operator-config\") pod \"servicemesh-operator3-57f65f65fb-84drr\" (UID: \"fc2d3adb-1049-4977-9ae9-ec9285b4c40c\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-84drr" Jun 15 03:16:36.932128 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:36.932067 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/fc2d3adb-1049-4977-9ae9-ec9285b4c40c-operator-config\") pod \"servicemesh-operator3-57f65f65fb-84drr\" (UID: \"fc2d3adb-1049-4977-9ae9-ec9285b4c40c\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-84drr" Jun 15 03:16:36.932310 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:36.932170 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn2b6\" (UniqueName: \"kubernetes.io/projected/fc2d3adb-1049-4977-9ae9-ec9285b4c40c-kube-api-access-mn2b6\") pod \"servicemesh-operator3-57f65f65fb-84drr\" (UID: \"fc2d3adb-1049-4977-9ae9-ec9285b4c40c\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-84drr" Jun 15 03:16:36.934728 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:36.934706 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/fc2d3adb-1049-4977-9ae9-ec9285b4c40c-operator-config\") pod \"servicemesh-operator3-57f65f65fb-84drr\" (UID: \"fc2d3adb-1049-4977-9ae9-ec9285b4c40c\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-84drr" Jun 15 03:16:36.942415 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:36.942386 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn2b6\" (UniqueName: \"kubernetes.io/projected/fc2d3adb-1049-4977-9ae9-ec9285b4c40c-kube-api-access-mn2b6\") pod \"servicemesh-operator3-57f65f65fb-84drr\" (UID: \"fc2d3adb-1049-4977-9ae9-ec9285b4c40c\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-84drr" Jun 15 03:16:37.023815 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:37.023775 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-84drr" Jun 15 03:16:37.202739 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:37.202711 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-84drr"] Jun 15 03:16:37.204797 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:16:37.204763 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc2d3adb_1049_4977_9ae9_ec9285b4c40c.slice/crio-2d45142788d153eeba40cf82dc272e75f4fbae233711b2677482fce6a210aa6a WatchSource:0}: Error finding container 2d45142788d153eeba40cf82dc272e75f4fbae233711b2677482fce6a210aa6a: Status 404 returned error can't find the container with id 2d45142788d153eeba40cf82dc272e75f4fbae233711b2677482fce6a210aa6a Jun 15 03:16:38.055477 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.055444 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-694cb6497d-qpx9v"] Jun 15 03:16:38.058832 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.058813 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-694cb6497d-qpx9v" Jun 15 03:16:38.063403 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.063374 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 15 03:16:38.064862 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.064845 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-485sd\"" Jun 15 03:16:38.072750 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.072726 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-694cb6497d-qpx9v"] Jun 15 03:16:38.123734 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.123697 2560 generic.go:356] "Generic (PLEG): container finished" podID="ec2e70c7-0439-43e6-8043-093b956877aa" containerID="26259e1dc5c8cf528cc02c5bdca5d8ad17b36859ceaceffa898c6c402324e6ce" exitCode=0 Jun 15 03:16:38.123917 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.123783 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" event={"ID":"ec2e70c7-0439-43e6-8043-093b956877aa","Type":"ContainerDied","Data":"26259e1dc5c8cf528cc02c5bdca5d8ad17b36859ceaceffa898c6c402324e6ce"} Jun 15 03:16:38.125547 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.125521 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-84drr" event={"ID":"fc2d3adb-1049-4977-9ae9-ec9285b4c40c","Type":"ContainerStarted","Data":"2d45142788d153eeba40cf82dc272e75f4fbae233711b2677482fce6a210aa6a"} Jun 15 03:16:38.143297 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.143269 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/7749efc3-a41d-4aac-a75d-f28b4f1b9ec0-redirect-config\") pod \"dashboard-redirect-694cb6497d-qpx9v\" (UID: \"7749efc3-a41d-4aac-a75d-f28b4f1b9ec0\") " pod="opendatahub/dashboard-redirect-694cb6497d-qpx9v" Jun 15 03:16:38.143441 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.143308 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qch5w\" (UniqueName: \"kubernetes.io/projected/7749efc3-a41d-4aac-a75d-f28b4f1b9ec0-kube-api-access-qch5w\") pod \"dashboard-redirect-694cb6497d-qpx9v\" (UID: \"7749efc3-a41d-4aac-a75d-f28b4f1b9ec0\") " pod="opendatahub/dashboard-redirect-694cb6497d-qpx9v" Jun 15 03:16:38.244620 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.244580 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/7749efc3-a41d-4aac-a75d-f28b4f1b9ec0-redirect-config\") pod \"dashboard-redirect-694cb6497d-qpx9v\" (UID: \"7749efc3-a41d-4aac-a75d-f28b4f1b9ec0\") " pod="opendatahub/dashboard-redirect-694cb6497d-qpx9v" Jun 15 03:16:38.244797 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.244678 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qch5w\" (UniqueName: \"kubernetes.io/projected/7749efc3-a41d-4aac-a75d-f28b4f1b9ec0-kube-api-access-qch5w\") pod \"dashboard-redirect-694cb6497d-qpx9v\" (UID: \"7749efc3-a41d-4aac-a75d-f28b4f1b9ec0\") " pod="opendatahub/dashboard-redirect-694cb6497d-qpx9v" Jun 15 03:16:38.245363 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.245340 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/7749efc3-a41d-4aac-a75d-f28b4f1b9ec0-redirect-config\") pod \"dashboard-redirect-694cb6497d-qpx9v\" (UID: \"7749efc3-a41d-4aac-a75d-f28b4f1b9ec0\") " pod="opendatahub/dashboard-redirect-694cb6497d-qpx9v" Jun 15 03:16:38.255764 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.255736 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qch5w\" (UniqueName: \"kubernetes.io/projected/7749efc3-a41d-4aac-a75d-f28b4f1b9ec0-kube-api-access-qch5w\") pod \"dashboard-redirect-694cb6497d-qpx9v\" (UID: \"7749efc3-a41d-4aac-a75d-f28b4f1b9ec0\") " pod="opendatahub/dashboard-redirect-694cb6497d-qpx9v" Jun 15 03:16:38.369953 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.369915 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-694cb6497d-qpx9v" Jun 15 03:16:38.519760 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:38.519730 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-694cb6497d-qpx9v"] Jun 15 03:16:38.521069 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:16:38.521040 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7749efc3_a41d_4aac_a75d_f28b4f1b9ec0.slice/crio-c23bbb142c8bf0297684151c679b05e3bd11c56a19435d4c1f3400672919281b WatchSource:0}: Error finding container c23bbb142c8bf0297684151c679b05e3bd11c56a19435d4c1f3400672919281b: Status 404 returned error can't find the container with id c23bbb142c8bf0297684151c679b05e3bd11c56a19435d4c1f3400672919281b Jun 15 03:16:39.136892 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:39.136852 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-694cb6497d-qpx9v" event={"ID":"7749efc3-a41d-4aac-a75d-f28b4f1b9ec0","Type":"ContainerStarted","Data":"c23bbb142c8bf0297684151c679b05e3bd11c56a19435d4c1f3400672919281b"} Jun 15 03:16:39.139734 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:39.139631 2560 generic.go:356] "Generic (PLEG): container finished" podID="ec2e70c7-0439-43e6-8043-093b956877aa" containerID="9b1844df0402eedac8d9b9409637d65749670830a91f2f42282c44d8db8c2aa8" exitCode=0 Jun 15 03:16:39.139918 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:39.139716 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" event={"ID":"ec2e70c7-0439-43e6-8043-093b956877aa","Type":"ContainerDied","Data":"9b1844df0402eedac8d9b9409637d65749670830a91f2f42282c44d8db8c2aa8"} Jun 15 03:16:40.322060 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:40.322036 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" Jun 15 03:16:40.468338 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:40.468240 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec2e70c7-0439-43e6-8043-093b956877aa-util\") pod \"ec2e70c7-0439-43e6-8043-093b956877aa\" (UID: \"ec2e70c7-0439-43e6-8043-093b956877aa\") " Jun 15 03:16:40.468338 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:40.468286 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldv9x\" (UniqueName: \"kubernetes.io/projected/ec2e70c7-0439-43e6-8043-093b956877aa-kube-api-access-ldv9x\") pod \"ec2e70c7-0439-43e6-8043-093b956877aa\" (UID: \"ec2e70c7-0439-43e6-8043-093b956877aa\") " Jun 15 03:16:40.468520 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:40.468372 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec2e70c7-0439-43e6-8043-093b956877aa-bundle\") pod \"ec2e70c7-0439-43e6-8043-093b956877aa\" (UID: \"ec2e70c7-0439-43e6-8043-093b956877aa\") " Jun 15 03:16:40.469622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:40.469590 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec2e70c7-0439-43e6-8043-093b956877aa-bundle" (OuterVolumeSpecName: "bundle") pod "ec2e70c7-0439-43e6-8043-093b956877aa" (UID: "ec2e70c7-0439-43e6-8043-093b956877aa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:16:40.471129 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:40.471105 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec2e70c7-0439-43e6-8043-093b956877aa-kube-api-access-ldv9x" (OuterVolumeSpecName: "kube-api-access-ldv9x") pod "ec2e70c7-0439-43e6-8043-093b956877aa" (UID: "ec2e70c7-0439-43e6-8043-093b956877aa"). InnerVolumeSpecName "kube-api-access-ldv9x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:16:40.476069 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:40.476034 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec2e70c7-0439-43e6-8043-093b956877aa-util" (OuterVolumeSpecName: "util") pod "ec2e70c7-0439-43e6-8043-093b956877aa" (UID: "ec2e70c7-0439-43e6-8043-093b956877aa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:16:40.569070 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:40.569032 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec2e70c7-0439-43e6-8043-093b956877aa-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:16:40.569070 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:40.569072 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldv9x\" (UniqueName: \"kubernetes.io/projected/ec2e70c7-0439-43e6-8043-093b956877aa-kube-api-access-ldv9x\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:16:40.569333 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:40.569108 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec2e70c7-0439-43e6-8043-093b956877aa-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:16:41.152984 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:41.152960 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" Jun 15 03:16:41.153132 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:41.152963 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a7fprs" event={"ID":"ec2e70c7-0439-43e6-8043-093b956877aa","Type":"ContainerDied","Data":"21470c7a8488a1239b427490f73c18205fdd0d85f4686a7fa440ccf68317945a"} Jun 15 03:16:41.153132 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:41.153117 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21470c7a8488a1239b427490f73c18205fdd0d85f4686a7fa440ccf68317945a" Jun 15 03:16:42.159634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:42.159595 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-694cb6497d-qpx9v" event={"ID":"7749efc3-a41d-4aac-a75d-f28b4f1b9ec0","Type":"ContainerStarted","Data":"f2170b8b269c31d0a8ef71066afaec7d0e34d802b07db643e8b1cb2a6e139e3a"} Jun 15 03:16:42.161343 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:42.161322 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-84drr" event={"ID":"fc2d3adb-1049-4977-9ae9-ec9285b4c40c","Type":"ContainerStarted","Data":"e682b569e4503b9a264cd9380bbc3fd73b2d73eb3915b2348fe3c88828076f84"} Jun 15 03:16:42.161504 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:42.161492 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-84drr" Jun 15 03:16:42.177673 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:42.177630 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-694cb6497d-qpx9v" podStartSLOduration=1.520408294 podStartE2EDuration="4.177615512s" totalImagesPullingTime="2.657207218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:16:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:16:42.175531741 +0000 UTC m=+480.332145950" watchObservedRunningTime="2026-06-15 03:16:42.177615512 +0000 UTC m=+480.334229776" Jun 15 03:16:42.196359 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:42.196294 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-57f65f65fb-84drr" podStartSLOduration=2.280001494 podStartE2EDuration="6.196276546s" totalImagesPullingTime="3.916275052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:16:42.193071603 +0000 UTC m=+480.349685811" watchObservedRunningTime="2026-06-15 03:16:42.196276546 +0000 UTC m=+480.352890753" Jun 15 03:16:51.751804 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.751770 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l"] Jun 15 03:16:51.752263 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.752162 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec2e70c7-0439-43e6-8043-093b956877aa" containerName="util" Jun 15 03:16:51.752263 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.752175 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec2e70c7-0439-43e6-8043-093b956877aa" containerName="util" Jun 15 03:16:51.752263 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.752194 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec2e70c7-0439-43e6-8043-093b956877aa" containerName="pull" Jun 15 03:16:51.752263 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.752200 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec2e70c7-0439-43e6-8043-093b956877aa" containerName="pull" Jun 15 03:16:51.752263 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.752211 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec2e70c7-0439-43e6-8043-093b956877aa" containerName="extract" Jun 15 03:16:51.752263 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.752217 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec2e70c7-0439-43e6-8043-093b956877aa" containerName="extract" Jun 15 03:16:51.752459 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.752273 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec2e70c7-0439-43e6-8043-093b956877aa" containerName="extract" Jun 15 03:16:51.757059 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.757042 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.759855 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.759833 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 15 03:16:51.759968 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.759832 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 15 03:16:51.759968 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.759832 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-sfxb2\"" Jun 15 03:16:51.759968 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.759868 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 15 03:16:51.760127 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.760104 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 15 03:16:51.762040 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.761966 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0429504b-413e-45fb-ae04-8345e31b1f31-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.762040 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.761999 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0429504b-413e-45fb-ae04-8345e31b1f31-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.762238 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.762055 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0429504b-413e-45fb-ae04-8345e31b1f31-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.762238 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.762100 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nspbt\" (UniqueName: \"kubernetes.io/projected/0429504b-413e-45fb-ae04-8345e31b1f31-kube-api-access-nspbt\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.762238 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.762138 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0429504b-413e-45fb-ae04-8345e31b1f31-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.762238 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.762173 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0429504b-413e-45fb-ae04-8345e31b1f31-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.762238 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.762209 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0429504b-413e-45fb-ae04-8345e31b1f31-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.764761 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.764741 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l"] Jun 15 03:16:51.862954 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.862919 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0429504b-413e-45fb-ae04-8345e31b1f31-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.863189 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.862967 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0429504b-413e-45fb-ae04-8345e31b1f31-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.863189 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.863011 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0429504b-413e-45fb-ae04-8345e31b1f31-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.863189 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.863031 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0429504b-413e-45fb-ae04-8345e31b1f31-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.863401 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.863194 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0429504b-413e-45fb-ae04-8345e31b1f31-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.863401 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.863231 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nspbt\" (UniqueName: \"kubernetes.io/projected/0429504b-413e-45fb-ae04-8345e31b1f31-kube-api-access-nspbt\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.863401 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.863275 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0429504b-413e-45fb-ae04-8345e31b1f31-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.863744 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.863713 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0429504b-413e-45fb-ae04-8345e31b1f31-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.865454 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.865428 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0429504b-413e-45fb-ae04-8345e31b1f31-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.865577 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.865559 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0429504b-413e-45fb-ae04-8345e31b1f31-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.865892 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.865871 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0429504b-413e-45fb-ae04-8345e31b1f31-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.865955 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.865898 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0429504b-413e-45fb-ae04-8345e31b1f31-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.871747 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.871727 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nspbt\" (UniqueName: \"kubernetes.io/projected/0429504b-413e-45fb-ae04-8345e31b1f31-kube-api-access-nspbt\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:51.871857 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:51.871840 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0429504b-413e-45fb-ae04-8345e31b1f31-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-l7p2l\" (UID: \"0429504b-413e-45fb-ae04-8345e31b1f31\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:52.067174 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:52.067137 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:52.205167 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:52.205140 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l"] Jun 15 03:16:52.207364 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:16:52.207332 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0429504b_413e_45fb_ae04_8345e31b1f31.slice/crio-2ce58a53207bcd5a2f401dd9de326ba164c5429cff02ef28034829021415def4 WatchSource:0}: Error finding container 2ce58a53207bcd5a2f401dd9de326ba164c5429cff02ef28034829021415def4: Status 404 returned error can't find the container with id 2ce58a53207bcd5a2f401dd9de326ba164c5429cff02ef28034829021415def4 Jun 15 03:16:52.223401 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:52.223371 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" event={"ID":"0429504b-413e-45fb-ae04-8345e31b1f31","Type":"ContainerStarted","Data":"2ce58a53207bcd5a2f401dd9de326ba164c5429cff02ef28034829021415def4"} Jun 15 03:16:53.169576 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:53.169543 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-84drr" Jun 15 03:16:56.063633 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:56.063584 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 15 03:16:56.063921 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:56.063661 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 15 03:16:56.248290 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:56.248254 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" event={"ID":"0429504b-413e-45fb-ae04-8345e31b1f31","Type":"ContainerStarted","Data":"f3e3a9f7ebcc15621983f40a5ebd3c5352788420286d978065e92a1bb946af69"} Jun 15 03:16:56.248452 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:56.248326 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:16:56.271046 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:56.270996 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" podStartSLOduration=1.417119161 podStartE2EDuration="5.270980724s" totalImagesPullingTime="3.853861563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:16:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:16:56.267515516 +0000 UTC m=+494.424129782" watchObservedRunningTime="2026-06-15 03:16:56.270980724 +0000 UTC m=+494.427594932" Jun 15 03:16:57.256803 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:57.256758 2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-75c67f8887-l7p2l container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 15 03:16:57.257215 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:16:57.256833 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" podUID="0429504b-413e-45fb-ae04-8345e31b1f31" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 15 03:17:00.256694 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:00.256659 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-l7p2l" Jun 15 03:17:01.088510 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:01.088472 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-69fc866d76-dbmcd" Jun 15 03:17:03.095308 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:03.095271 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-xmklr" Jun 15 03:17:32.790738 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:32.790657 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9"] Jun 15 03:17:32.798151 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:32.798132 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" Jun 15 03:17:32.801458 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:32.801434 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 15 03:17:32.801776 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:32.801456 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bkprt\"" Jun 15 03:17:32.801907 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:32.801533 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 15 03:17:32.803412 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:32.803391 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9"] Jun 15 03:17:32.825104 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:32.825054 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9\" (UID: \"5bf52ff2-469e-4c84-ae4c-e18b0a33f139\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" Jun 15 03:17:32.825213 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:32.825150 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9\" (UID: \"5bf52ff2-469e-4c84-ae4c-e18b0a33f139\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" Jun 15 03:17:32.825259 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:32.825228 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j222s\" (UniqueName: \"kubernetes.io/projected/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-kube-api-access-j222s\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9\" (UID: \"5bf52ff2-469e-4c84-ae4c-e18b0a33f139\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" Jun 15 03:17:32.925556 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:32.925522 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9\" (UID: \"5bf52ff2-469e-4c84-ae4c-e18b0a33f139\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" Jun 15 03:17:32.925690 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:32.925582 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9\" (UID: \"5bf52ff2-469e-4c84-ae4c-e18b0a33f139\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" Jun 15 03:17:32.925690 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:32.925644 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j222s\" (UniqueName: \"kubernetes.io/projected/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-kube-api-access-j222s\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9\" (UID: \"5bf52ff2-469e-4c84-ae4c-e18b0a33f139\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" Jun 15 03:17:32.925866 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:32.925844 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9\" (UID: \"5bf52ff2-469e-4c84-ae4c-e18b0a33f139\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" Jun 15 03:17:32.925931 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:32.925901 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9\" (UID: \"5bf52ff2-469e-4c84-ae4c-e18b0a33f139\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" Jun 15 03:17:32.934641 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:32.934623 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j222s\" (UniqueName: \"kubernetes.io/projected/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-kube-api-access-j222s\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9\" (UID: \"5bf52ff2-469e-4c84-ae4c-e18b0a33f139\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" Jun 15 03:17:33.108657 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.108590 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" Jun 15 03:17:33.238241 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.238218 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9"] Jun 15 03:17:33.240121 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:17:33.240071 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf52ff2_469e_4c84_ae4c_e18b0a33f139.slice/crio-94602f8993dc20311a150f6b8e3650233966683c592e890fc15b71d19418a99f WatchSource:0}: Error finding container 94602f8993dc20311a150f6b8e3650233966683c592e890fc15b71d19418a99f: Status 404 returned error can't find the container with id 94602f8993dc20311a150f6b8e3650233966683c592e890fc15b71d19418a99f Jun 15 03:17:33.389858 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.389795 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl"] Jun 15 03:17:33.393453 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.393437 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" Jun 15 03:17:33.400247 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.400224 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl"] Jun 15 03:17:33.428783 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.428761 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41832d93-7b55-4125-8f59-b5ea5a61fa73-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl\" (UID: \"41832d93-7b55-4125-8f59-b5ea5a61fa73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" Jun 15 03:17:33.428892 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.428798 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41832d93-7b55-4125-8f59-b5ea5a61fa73-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl\" (UID: \"41832d93-7b55-4125-8f59-b5ea5a61fa73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" Jun 15 03:17:33.428892 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.428868 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmdl6\" (UniqueName: \"kubernetes.io/projected/41832d93-7b55-4125-8f59-b5ea5a61fa73-kube-api-access-xmdl6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl\" (UID: \"41832d93-7b55-4125-8f59-b5ea5a61fa73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" Jun 15 03:17:33.433482 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.433456 2560 generic.go:356] "Generic (PLEG): container finished" podID="5bf52ff2-469e-4c84-ae4c-e18b0a33f139" containerID="b6cbf09303a070375326229ffc71089e465b2b71893b01ddf646ae4bf3b5f9c4" exitCode=0 Jun 15 03:17:33.433584 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.433498 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" event={"ID":"5bf52ff2-469e-4c84-ae4c-e18b0a33f139","Type":"ContainerDied","Data":"b6cbf09303a070375326229ffc71089e465b2b71893b01ddf646ae4bf3b5f9c4"} Jun 15 03:17:33.433584 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.433549 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" event={"ID":"5bf52ff2-469e-4c84-ae4c-e18b0a33f139","Type":"ContainerStarted","Data":"94602f8993dc20311a150f6b8e3650233966683c592e890fc15b71d19418a99f"} Jun 15 03:17:33.529603 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.529571 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41832d93-7b55-4125-8f59-b5ea5a61fa73-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl\" (UID: \"41832d93-7b55-4125-8f59-b5ea5a61fa73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" Jun 15 03:17:33.529769 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.529628 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmdl6\" (UniqueName: \"kubernetes.io/projected/41832d93-7b55-4125-8f59-b5ea5a61fa73-kube-api-access-xmdl6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl\" (UID: \"41832d93-7b55-4125-8f59-b5ea5a61fa73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" Jun 15 03:17:33.529769 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.529666 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41832d93-7b55-4125-8f59-b5ea5a61fa73-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl\" (UID: \"41832d93-7b55-4125-8f59-b5ea5a61fa73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" Jun 15 03:17:33.530002 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.529983 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41832d93-7b55-4125-8f59-b5ea5a61fa73-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl\" (UID: \"41832d93-7b55-4125-8f59-b5ea5a61fa73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" Jun 15 03:17:33.530069 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.530009 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41832d93-7b55-4125-8f59-b5ea5a61fa73-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl\" (UID: \"41832d93-7b55-4125-8f59-b5ea5a61fa73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" Jun 15 03:17:33.537914 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.537890 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmdl6\" (UniqueName: \"kubernetes.io/projected/41832d93-7b55-4125-8f59-b5ea5a61fa73-kube-api-access-xmdl6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl\" (UID: \"41832d93-7b55-4125-8f59-b5ea5a61fa73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" Jun 15 03:17:33.718530 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.718449 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" Jun 15 03:17:33.847382 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:33.847356 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl"] Jun 15 03:17:33.848518 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:17:33.848491 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41832d93_7b55_4125_8f59_b5ea5a61fa73.slice/crio-e4fd1269a4324b9199b17e53a0283d7d83e6afcaf08ef5e53318e34c5cc7e1a8 WatchSource:0}: Error finding container e4fd1269a4324b9199b17e53a0283d7d83e6afcaf08ef5e53318e34c5cc7e1a8: Status 404 returned error can't find the container with id e4fd1269a4324b9199b17e53a0283d7d83e6afcaf08ef5e53318e34c5cc7e1a8 Jun 15 03:17:34.000632 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.000548 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk"] Jun 15 03:17:34.005013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.004990 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" Jun 15 03:17:34.009990 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.009936 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk"] Jun 15 03:17:34.033413 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.033380 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhxc5\" (UniqueName: \"kubernetes.io/projected/b5ee597b-dd27-42c6-8351-5ddefd113f9f-kube-api-access-lhxc5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk\" (UID: \"b5ee597b-dd27-42c6-8351-5ddefd113f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" Jun 15 03:17:34.033528 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.033416 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5ee597b-dd27-42c6-8351-5ddefd113f9f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk\" (UID: \"b5ee597b-dd27-42c6-8351-5ddefd113f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" Jun 15 03:17:34.033528 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.033440 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5ee597b-dd27-42c6-8351-5ddefd113f9f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk\" (UID: \"b5ee597b-dd27-42c6-8351-5ddefd113f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" Jun 15 03:17:34.134445 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.134416 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhxc5\" (UniqueName: \"kubernetes.io/projected/b5ee597b-dd27-42c6-8351-5ddefd113f9f-kube-api-access-lhxc5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk\" (UID: \"b5ee597b-dd27-42c6-8351-5ddefd113f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" Jun 15 03:17:34.134609 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.134460 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5ee597b-dd27-42c6-8351-5ddefd113f9f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk\" (UID: \"b5ee597b-dd27-42c6-8351-5ddefd113f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" Jun 15 03:17:34.134609 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.134486 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5ee597b-dd27-42c6-8351-5ddefd113f9f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk\" (UID: \"b5ee597b-dd27-42c6-8351-5ddefd113f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" Jun 15 03:17:34.134790 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.134775 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5ee597b-dd27-42c6-8351-5ddefd113f9f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk\" (UID: \"b5ee597b-dd27-42c6-8351-5ddefd113f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" Jun 15 03:17:34.134826 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.134805 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5ee597b-dd27-42c6-8351-5ddefd113f9f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk\" (UID: \"b5ee597b-dd27-42c6-8351-5ddefd113f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" Jun 15 03:17:34.142988 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.142959 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhxc5\" (UniqueName: \"kubernetes.io/projected/b5ee597b-dd27-42c6-8351-5ddefd113f9f-kube-api-access-lhxc5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk\" (UID: \"b5ee597b-dd27-42c6-8351-5ddefd113f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" Jun 15 03:17:34.317456 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.317427 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" Jun 15 03:17:34.394130 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.394099 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq"] Jun 15 03:17:34.399171 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.399021 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" Jun 15 03:17:34.406864 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.406838 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq"] Jun 15 03:17:34.436445 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.436415 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71d99a6d-64e4-406e-be19-7cb8f58fe719-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq\" (UID: \"71d99a6d-64e4-406e-be19-7cb8f58fe719\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" Jun 15 03:17:34.436607 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.436468 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71d99a6d-64e4-406e-be19-7cb8f58fe719-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq\" (UID: \"71d99a6d-64e4-406e-be19-7cb8f58fe719\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" Jun 15 03:17:34.436607 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.436525 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgrx7\" (UniqueName: \"kubernetes.io/projected/71d99a6d-64e4-406e-be19-7cb8f58fe719-kube-api-access-tgrx7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq\" (UID: \"71d99a6d-64e4-406e-be19-7cb8f58fe719\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" Jun 15 03:17:34.444037 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.444006 2560 generic.go:356] "Generic (PLEG): container finished" podID="41832d93-7b55-4125-8f59-b5ea5a61fa73" containerID="0353ac7c7bc58bc8051191d3789d3040612f921077b4a7227d03c842ca683397" exitCode=0 Jun 15 03:17:34.444184 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.444158 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" event={"ID":"41832d93-7b55-4125-8f59-b5ea5a61fa73","Type":"ContainerDied","Data":"0353ac7c7bc58bc8051191d3789d3040612f921077b4a7227d03c842ca683397"} Jun 15 03:17:34.444261 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.444219 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" event={"ID":"41832d93-7b55-4125-8f59-b5ea5a61fa73","Type":"ContainerStarted","Data":"e4fd1269a4324b9199b17e53a0283d7d83e6afcaf08ef5e53318e34c5cc7e1a8"} Jun 15 03:17:34.445983 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.445958 2560 generic.go:356] "Generic (PLEG): container finished" podID="5bf52ff2-469e-4c84-ae4c-e18b0a33f139" containerID="2cb08f30038d8570351dd1d3a432f9e4edd28b81d8c2a46ebb3457e96bfb4b5c" exitCode=0 Jun 15 03:17:34.446074 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.446030 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" event={"ID":"5bf52ff2-469e-4c84-ae4c-e18b0a33f139","Type":"ContainerDied","Data":"2cb08f30038d8570351dd1d3a432f9e4edd28b81d8c2a46ebb3457e96bfb4b5c"} Jun 15 03:17:34.451988 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.451970 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk"] Jun 15 03:17:34.453705 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:17:34.453684 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5ee597b_dd27_42c6_8351_5ddefd113f9f.slice/crio-9d9c5c54c57e31430257da8b2c6bdcf9596e1daacfe7901f756bbd57b16f4a52 WatchSource:0}: Error finding container 9d9c5c54c57e31430257da8b2c6bdcf9596e1daacfe7901f756bbd57b16f4a52: Status 404 returned error can't find the container with id 9d9c5c54c57e31430257da8b2c6bdcf9596e1daacfe7901f756bbd57b16f4a52 Jun 15 03:17:34.537510 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.537474 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71d99a6d-64e4-406e-be19-7cb8f58fe719-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq\" (UID: \"71d99a6d-64e4-406e-be19-7cb8f58fe719\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" Jun 15 03:17:34.537670 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.537612 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgrx7\" (UniqueName: \"kubernetes.io/projected/71d99a6d-64e4-406e-be19-7cb8f58fe719-kube-api-access-tgrx7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq\" (UID: \"71d99a6d-64e4-406e-be19-7cb8f58fe719\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" Jun 15 03:17:34.537774 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.537742 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71d99a6d-64e4-406e-be19-7cb8f58fe719-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq\" (UID: \"71d99a6d-64e4-406e-be19-7cb8f58fe719\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" Jun 15 03:17:34.538179 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.538152 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71d99a6d-64e4-406e-be19-7cb8f58fe719-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq\" (UID: \"71d99a6d-64e4-406e-be19-7cb8f58fe719\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" Jun 15 03:17:34.538340 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.538246 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71d99a6d-64e4-406e-be19-7cb8f58fe719-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq\" (UID: \"71d99a6d-64e4-406e-be19-7cb8f58fe719\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" Jun 15 03:17:34.546538 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.546513 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgrx7\" (UniqueName: \"kubernetes.io/projected/71d99a6d-64e4-406e-be19-7cb8f58fe719-kube-api-access-tgrx7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq\" (UID: \"71d99a6d-64e4-406e-be19-7cb8f58fe719\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" Jun 15 03:17:34.712533 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.712501 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" Jun 15 03:17:34.844251 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:34.844223 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq"] Jun 15 03:17:34.846030 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:17:34.845996 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71d99a6d_64e4_406e_be19_7cb8f58fe719.slice/crio-8dc096428c2de894847cb5ecb0691d8c1a29a19837ba4b4ee62733b667a31eb2 WatchSource:0}: Error finding container 8dc096428c2de894847cb5ecb0691d8c1a29a19837ba4b4ee62733b667a31eb2: Status 404 returned error can't find the container with id 8dc096428c2de894847cb5ecb0691d8c1a29a19837ba4b4ee62733b667a31eb2 Jun 15 03:17:35.454220 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:35.454128 2560 generic.go:356] "Generic (PLEG): container finished" podID="5bf52ff2-469e-4c84-ae4c-e18b0a33f139" containerID="33a0902fa6208c2504db4aa462474893c060ad761e11c7b97fa9ba641b2232b6" exitCode=0 Jun 15 03:17:35.454220 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:35.454210 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" event={"ID":"5bf52ff2-469e-4c84-ae4c-e18b0a33f139","Type":"ContainerDied","Data":"33a0902fa6208c2504db4aa462474893c060ad761e11c7b97fa9ba641b2232b6"} Jun 15 03:17:35.455708 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:35.455686 2560 generic.go:356] "Generic (PLEG): container finished" podID="b5ee597b-dd27-42c6-8351-5ddefd113f9f" containerID="6d395aff4842e0590e306a4b157f8c9e382ac0b34dfeb4d5642bab1dacd54636" exitCode=0 Jun 15 03:17:35.455831 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:35.455773 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" event={"ID":"b5ee597b-dd27-42c6-8351-5ddefd113f9f","Type":"ContainerDied","Data":"6d395aff4842e0590e306a4b157f8c9e382ac0b34dfeb4d5642bab1dacd54636"} Jun 15 03:17:35.455831 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:35.455816 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" event={"ID":"b5ee597b-dd27-42c6-8351-5ddefd113f9f","Type":"ContainerStarted","Data":"9d9c5c54c57e31430257da8b2c6bdcf9596e1daacfe7901f756bbd57b16f4a52"} Jun 15 03:17:35.457625 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:35.457605 2560 generic.go:356] "Generic (PLEG): container finished" podID="71d99a6d-64e4-406e-be19-7cb8f58fe719" containerID="abc957e27e0bc9295b207da1ee73e972eeedab75a61dc9bdf2fe3f27b73d632a" exitCode=0 Jun 15 03:17:35.457745 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:35.457690 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" event={"ID":"71d99a6d-64e4-406e-be19-7cb8f58fe719","Type":"ContainerDied","Data":"abc957e27e0bc9295b207da1ee73e972eeedab75a61dc9bdf2fe3f27b73d632a"} Jun 15 03:17:35.457745 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:35.457735 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" event={"ID":"71d99a6d-64e4-406e-be19-7cb8f58fe719","Type":"ContainerStarted","Data":"8dc096428c2de894847cb5ecb0691d8c1a29a19837ba4b4ee62733b667a31eb2"} Jun 15 03:17:35.459807 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:35.459782 2560 generic.go:356] "Generic (PLEG): container finished" podID="41832d93-7b55-4125-8f59-b5ea5a61fa73" containerID="cd8ec2c1817644d69bdcd8caf417acccaa1adb78d14c281193cff1f29fc6e065" exitCode=0 Jun 15 03:17:35.459904 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:35.459811 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" event={"ID":"41832d93-7b55-4125-8f59-b5ea5a61fa73","Type":"ContainerDied","Data":"cd8ec2c1817644d69bdcd8caf417acccaa1adb78d14c281193cff1f29fc6e065"} Jun 15 03:17:36.467808 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.467776 2560 generic.go:356] "Generic (PLEG): container finished" podID="41832d93-7b55-4125-8f59-b5ea5a61fa73" containerID="47a1aa280bb933bd46440411afd9248bd80ad4d4e46c64dbd7d6a497b7a70b88" exitCode=0 Jun 15 03:17:36.468145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.467833 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" event={"ID":"41832d93-7b55-4125-8f59-b5ea5a61fa73","Type":"ContainerDied","Data":"47a1aa280bb933bd46440411afd9248bd80ad4d4e46c64dbd7d6a497b7a70b88"} Jun 15 03:17:36.469549 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.469529 2560 generic.go:356] "Generic (PLEG): container finished" podID="b5ee597b-dd27-42c6-8351-5ddefd113f9f" containerID="7854aed34189deff0f64d8d961df543e0742998efb4a86091f4ba907eca1f506" exitCode=0 Jun 15 03:17:36.469640 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.469603 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" event={"ID":"b5ee597b-dd27-42c6-8351-5ddefd113f9f","Type":"ContainerDied","Data":"7854aed34189deff0f64d8d961df543e0742998efb4a86091f4ba907eca1f506"} Jun 15 03:17:36.471414 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.471394 2560 generic.go:356] "Generic (PLEG): container finished" podID="71d99a6d-64e4-406e-be19-7cb8f58fe719" containerID="05e37968d00fb0f3271857b52051f163208795363a3620105205a6d82c4c1c34" exitCode=0 Jun 15 03:17:36.471507 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.471462 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" event={"ID":"71d99a6d-64e4-406e-be19-7cb8f58fe719","Type":"ContainerDied","Data":"05e37968d00fb0f3271857b52051f163208795363a3620105205a6d82c4c1c34"} Jun 15 03:17:36.605165 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.605137 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" Jun 15 03:17:36.654344 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.654316 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-util\") pod \"5bf52ff2-469e-4c84-ae4c-e18b0a33f139\" (UID: \"5bf52ff2-469e-4c84-ae4c-e18b0a33f139\") " Jun 15 03:17:36.654473 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.654361 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j222s\" (UniqueName: \"kubernetes.io/projected/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-kube-api-access-j222s\") pod \"5bf52ff2-469e-4c84-ae4c-e18b0a33f139\" (UID: \"5bf52ff2-469e-4c84-ae4c-e18b0a33f139\") " Jun 15 03:17:36.654473 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.654450 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-bundle\") pod \"5bf52ff2-469e-4c84-ae4c-e18b0a33f139\" (UID: \"5bf52ff2-469e-4c84-ae4c-e18b0a33f139\") " Jun 15 03:17:36.655197 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.655073 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-bundle" (OuterVolumeSpecName: "bundle") pod "5bf52ff2-469e-4c84-ae4c-e18b0a33f139" (UID: "5bf52ff2-469e-4c84-ae4c-e18b0a33f139"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:17:36.656467 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.656445 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-kube-api-access-j222s" (OuterVolumeSpecName: "kube-api-access-j222s") pod "5bf52ff2-469e-4c84-ae4c-e18b0a33f139" (UID: "5bf52ff2-469e-4c84-ae4c-e18b0a33f139"). InnerVolumeSpecName "kube-api-access-j222s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:17:36.659579 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.659556 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-util" (OuterVolumeSpecName: "util") pod "5bf52ff2-469e-4c84-ae4c-e18b0a33f139" (UID: "5bf52ff2-469e-4c84-ae4c-e18b0a33f139"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:17:36.755139 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.755038 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:17:36.755139 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.755063 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j222s\" (UniqueName: \"kubernetes.io/projected/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-kube-api-access-j222s\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:17:36.755139 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:36.755073 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bf52ff2-469e-4c84-ae4c-e18b0a33f139-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:17:37.479036 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.479006 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" Jun 15 03:17:37.479453 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.479001 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9" event={"ID":"5bf52ff2-469e-4c84-ae4c-e18b0a33f139","Type":"ContainerDied","Data":"94602f8993dc20311a150f6b8e3650233966683c592e890fc15b71d19418a99f"} Jun 15 03:17:37.479453 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.479145 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94602f8993dc20311a150f6b8e3650233966683c592e890fc15b71d19418a99f" Jun 15 03:17:37.480954 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.480930 2560 generic.go:356] "Generic (PLEG): container finished" podID="b5ee597b-dd27-42c6-8351-5ddefd113f9f" containerID="b961e819b4922ff4c7812989f0908dd79192a75197a3807155f7ce48261e7524" exitCode=0 Jun 15 03:17:37.481065 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.480973 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" event={"ID":"b5ee597b-dd27-42c6-8351-5ddefd113f9f","Type":"ContainerDied","Data":"b961e819b4922ff4c7812989f0908dd79192a75197a3807155f7ce48261e7524"} Jun 15 03:17:37.482804 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.482783 2560 generic.go:356] "Generic (PLEG): container finished" podID="71d99a6d-64e4-406e-be19-7cb8f58fe719" containerID="f838041a5aab6372eeed60875630aed0d9f2c41534ef0c9f9626b37533070ef2" exitCode=0 Jun 15 03:17:37.482905 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.482851 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" event={"ID":"71d99a6d-64e4-406e-be19-7cb8f58fe719","Type":"ContainerDied","Data":"f838041a5aab6372eeed60875630aed0d9f2c41534ef0c9f9626b37533070ef2"} Jun 15 03:17:37.614145 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.614123 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" Jun 15 03:17:37.662563 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.662533 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41832d93-7b55-4125-8f59-b5ea5a61fa73-bundle\") pod \"41832d93-7b55-4125-8f59-b5ea5a61fa73\" (UID: \"41832d93-7b55-4125-8f59-b5ea5a61fa73\") " Jun 15 03:17:37.662722 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.662578 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41832d93-7b55-4125-8f59-b5ea5a61fa73-util\") pod \"41832d93-7b55-4125-8f59-b5ea5a61fa73\" (UID: \"41832d93-7b55-4125-8f59-b5ea5a61fa73\") " Jun 15 03:17:37.662722 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.662646 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmdl6\" (UniqueName: \"kubernetes.io/projected/41832d93-7b55-4125-8f59-b5ea5a61fa73-kube-api-access-xmdl6\") pod \"41832d93-7b55-4125-8f59-b5ea5a61fa73\" (UID: \"41832d93-7b55-4125-8f59-b5ea5a61fa73\") " Jun 15 03:17:37.663060 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.663033 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41832d93-7b55-4125-8f59-b5ea5a61fa73-bundle" (OuterVolumeSpecName: "bundle") pod "41832d93-7b55-4125-8f59-b5ea5a61fa73" (UID: "41832d93-7b55-4125-8f59-b5ea5a61fa73"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:17:37.664734 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.664711 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41832d93-7b55-4125-8f59-b5ea5a61fa73-kube-api-access-xmdl6" (OuterVolumeSpecName: "kube-api-access-xmdl6") pod "41832d93-7b55-4125-8f59-b5ea5a61fa73" (UID: "41832d93-7b55-4125-8f59-b5ea5a61fa73"). InnerVolumeSpecName "kube-api-access-xmdl6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:17:37.667823 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.667782 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41832d93-7b55-4125-8f59-b5ea5a61fa73-util" (OuterVolumeSpecName: "util") pod "41832d93-7b55-4125-8f59-b5ea5a61fa73" (UID: "41832d93-7b55-4125-8f59-b5ea5a61fa73"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:17:37.763835 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.763755 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41832d93-7b55-4125-8f59-b5ea5a61fa73-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:17:37.763835 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.763783 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41832d93-7b55-4125-8f59-b5ea5a61fa73-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:17:37.763835 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:37.763792 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmdl6\" (UniqueName: \"kubernetes.io/projected/41832d93-7b55-4125-8f59-b5ea5a61fa73-kube-api-access-xmdl6\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:17:38.490046 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.490016 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" Jun 15 03:17:38.490046 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.490036 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl" event={"ID":"41832d93-7b55-4125-8f59-b5ea5a61fa73","Type":"ContainerDied","Data":"e4fd1269a4324b9199b17e53a0283d7d83e6afcaf08ef5e53318e34c5cc7e1a8"} Jun 15 03:17:38.490463 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.490120 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4fd1269a4324b9199b17e53a0283d7d83e6afcaf08ef5e53318e34c5cc7e1a8" Jun 15 03:17:38.658998 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.658971 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" Jun 15 03:17:38.661167 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.661150 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" Jun 15 03:17:38.670964 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.670942 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5ee597b-dd27-42c6-8351-5ddefd113f9f-bundle\") pod \"b5ee597b-dd27-42c6-8351-5ddefd113f9f\" (UID: \"b5ee597b-dd27-42c6-8351-5ddefd113f9f\") " Jun 15 03:17:38.671046 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.670987 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5ee597b-dd27-42c6-8351-5ddefd113f9f-util\") pod \"b5ee597b-dd27-42c6-8351-5ddefd113f9f\" (UID: \"b5ee597b-dd27-42c6-8351-5ddefd113f9f\") " Jun 15 03:17:38.671046 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.671015 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgrx7\" (UniqueName: \"kubernetes.io/projected/71d99a6d-64e4-406e-be19-7cb8f58fe719-kube-api-access-tgrx7\") pod \"71d99a6d-64e4-406e-be19-7cb8f58fe719\" (UID: \"71d99a6d-64e4-406e-be19-7cb8f58fe719\") " Jun 15 03:17:38.671183 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.671049 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71d99a6d-64e4-406e-be19-7cb8f58fe719-bundle\") pod \"71d99a6d-64e4-406e-be19-7cb8f58fe719\" (UID: \"71d99a6d-64e4-406e-be19-7cb8f58fe719\") " Jun 15 03:17:38.671183 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.671148 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71d99a6d-64e4-406e-be19-7cb8f58fe719-util\") pod \"71d99a6d-64e4-406e-be19-7cb8f58fe719\" (UID: \"71d99a6d-64e4-406e-be19-7cb8f58fe719\") " Jun 15 03:17:38.671291 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.671251 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhxc5\" (UniqueName: \"kubernetes.io/projected/b5ee597b-dd27-42c6-8351-5ddefd113f9f-kube-api-access-lhxc5\") pod \"b5ee597b-dd27-42c6-8351-5ddefd113f9f\" (UID: \"b5ee597b-dd27-42c6-8351-5ddefd113f9f\") " Jun 15 03:17:38.671680 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.671532 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5ee597b-dd27-42c6-8351-5ddefd113f9f-bundle" (OuterVolumeSpecName: "bundle") pod "b5ee597b-dd27-42c6-8351-5ddefd113f9f" (UID: "b5ee597b-dd27-42c6-8351-5ddefd113f9f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:17:38.671849 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.671829 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d99a6d-64e4-406e-be19-7cb8f58fe719-bundle" (OuterVolumeSpecName: "bundle") pod "71d99a6d-64e4-406e-be19-7cb8f58fe719" (UID: "71d99a6d-64e4-406e-be19-7cb8f58fe719"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:17:38.673242 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.673212 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71d99a6d-64e4-406e-be19-7cb8f58fe719-kube-api-access-tgrx7" (OuterVolumeSpecName: "kube-api-access-tgrx7") pod "71d99a6d-64e4-406e-be19-7cb8f58fe719" (UID: "71d99a6d-64e4-406e-be19-7cb8f58fe719"). InnerVolumeSpecName "kube-api-access-tgrx7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:17:38.673791 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.673748 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5ee597b-dd27-42c6-8351-5ddefd113f9f-kube-api-access-lhxc5" (OuterVolumeSpecName: "kube-api-access-lhxc5") pod "b5ee597b-dd27-42c6-8351-5ddefd113f9f" (UID: "b5ee597b-dd27-42c6-8351-5ddefd113f9f"). InnerVolumeSpecName "kube-api-access-lhxc5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:17:38.679415 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.679375 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d99a6d-64e4-406e-be19-7cb8f58fe719-util" (OuterVolumeSpecName: "util") pod "71d99a6d-64e4-406e-be19-7cb8f58fe719" (UID: "71d99a6d-64e4-406e-be19-7cb8f58fe719"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:17:38.679502 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.679398 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5ee597b-dd27-42c6-8351-5ddefd113f9f-util" (OuterVolumeSpecName: "util") pod "b5ee597b-dd27-42c6-8351-5ddefd113f9f" (UID: "b5ee597b-dd27-42c6-8351-5ddefd113f9f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:17:38.772317 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.772269 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgrx7\" (UniqueName: \"kubernetes.io/projected/71d99a6d-64e4-406e-be19-7cb8f58fe719-kube-api-access-tgrx7\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:17:38.772317 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.772314 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71d99a6d-64e4-406e-be19-7cb8f58fe719-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:17:38.772317 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.772324 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71d99a6d-64e4-406e-be19-7cb8f58fe719-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:17:38.772317 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.772332 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhxc5\" (UniqueName: \"kubernetes.io/projected/b5ee597b-dd27-42c6-8351-5ddefd113f9f-kube-api-access-lhxc5\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:17:38.772581 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.772344 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5ee597b-dd27-42c6-8351-5ddefd113f9f-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:17:38.772581 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:38.772353 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5ee597b-dd27-42c6-8351-5ddefd113f9f-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:17:39.496865 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:39.496834 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" Jun 15 03:17:39.496865 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:39.496857 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk" event={"ID":"b5ee597b-dd27-42c6-8351-5ddefd113f9f","Type":"ContainerDied","Data":"9d9c5c54c57e31430257da8b2c6bdcf9596e1daacfe7901f756bbd57b16f4a52"} Jun 15 03:17:39.497324 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:39.496904 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d9c5c54c57e31430257da8b2c6bdcf9596e1daacfe7901f756bbd57b16f4a52" Jun 15 03:17:39.498639 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:39.498617 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" event={"ID":"71d99a6d-64e4-406e-be19-7cb8f58fe719","Type":"ContainerDied","Data":"8dc096428c2de894847cb5ecb0691d8c1a29a19837ba4b4ee62733b667a31eb2"} Jun 15 03:17:39.498758 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:39.498651 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8dc096428c2de894847cb5ecb0691d8c1a29a19837ba4b4ee62733b667a31eb2" Jun 15 03:17:39.498758 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:39.498654 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq" Jun 15 03:17:57.698508 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698470 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mbfbx"] Jun 15 03:17:57.698885 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698837 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5ee597b-dd27-42c6-8351-5ddefd113f9f" containerName="extract" Jun 15 03:17:57.698885 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698849 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5ee597b-dd27-42c6-8351-5ddefd113f9f" containerName="extract" Jun 15 03:17:57.698885 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698860 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41832d93-7b55-4125-8f59-b5ea5a61fa73" containerName="pull" Jun 15 03:17:57.698885 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698882 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="41832d93-7b55-4125-8f59-b5ea5a61fa73" containerName="pull" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698892 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41832d93-7b55-4125-8f59-b5ea5a61fa73" containerName="util" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698898 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="41832d93-7b55-4125-8f59-b5ea5a61fa73" containerName="util" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698905 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bf52ff2-469e-4c84-ae4c-e18b0a33f139" containerName="pull" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698910 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bf52ff2-469e-4c84-ae4c-e18b0a33f139" containerName="pull" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698918 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71d99a6d-64e4-406e-be19-7cb8f58fe719" containerName="extract" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698925 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="71d99a6d-64e4-406e-be19-7cb8f58fe719" containerName="extract" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698933 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5ee597b-dd27-42c6-8351-5ddefd113f9f" containerName="pull" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698938 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5ee597b-dd27-42c6-8351-5ddefd113f9f" containerName="pull" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698943 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71d99a6d-64e4-406e-be19-7cb8f58fe719" containerName="pull" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698947 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="71d99a6d-64e4-406e-be19-7cb8f58fe719" containerName="pull" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698961 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bf52ff2-469e-4c84-ae4c-e18b0a33f139" containerName="util" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698965 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bf52ff2-469e-4c84-ae4c-e18b0a33f139" containerName="util" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698973 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71d99a6d-64e4-406e-be19-7cb8f58fe719" containerName="util" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698978 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="71d99a6d-64e4-406e-be19-7cb8f58fe719" containerName="util" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698986 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bf52ff2-469e-4c84-ae4c-e18b0a33f139" containerName="extract" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698991 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bf52ff2-469e-4c84-ae4c-e18b0a33f139" containerName="extract" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.698996 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5ee597b-dd27-42c6-8351-5ddefd113f9f" containerName="util" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.699001 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5ee597b-dd27-42c6-8351-5ddefd113f9f" containerName="util" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.699008 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41832d93-7b55-4125-8f59-b5ea5a61fa73" containerName="extract" Jun 15 03:17:57.699013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.699013 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="41832d93-7b55-4125-8f59-b5ea5a61fa73" containerName="extract" Jun 15 03:17:57.699569 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.699065 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="71d99a6d-64e4-406e-be19-7cb8f58fe719" containerName="extract" Jun 15 03:17:57.699569 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.699074 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="41832d93-7b55-4125-8f59-b5ea5a61fa73" containerName="extract" Jun 15 03:17:57.699569 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.699093 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5ee597b-dd27-42c6-8351-5ddefd113f9f" containerName="extract" Jun 15 03:17:57.699569 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.699100 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bf52ff2-469e-4c84-ae4c-e18b0a33f139" containerName="extract" Jun 15 03:17:57.702199 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.702175 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mbfbx" Jun 15 03:17:57.705091 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.705057 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 15 03:17:57.705207 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.705060 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 15 03:17:57.706238 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.706220 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 15 03:17:57.706238 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.706230 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-8scnl\"" Jun 15 03:17:57.712662 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.712639 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mbfbx"] Jun 15 03:17:57.718534 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.718511 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sdrh\" (UniqueName: \"kubernetes.io/projected/cd73322e-f25f-42b6-99ea-8f3080dc409a-kube-api-access-9sdrh\") pod \"dns-operator-controller-manager-864dd78c4c-mbfbx\" (UID: \"cd73322e-f25f-42b6-99ea-8f3080dc409a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mbfbx" Jun 15 03:17:57.819948 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.819920 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sdrh\" (UniqueName: \"kubernetes.io/projected/cd73322e-f25f-42b6-99ea-8f3080dc409a-kube-api-access-9sdrh\") pod \"dns-operator-controller-manager-864dd78c4c-mbfbx\" (UID: \"cd73322e-f25f-42b6-99ea-8f3080dc409a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mbfbx" Jun 15 03:17:57.829116 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:57.829077 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sdrh\" (UniqueName: \"kubernetes.io/projected/cd73322e-f25f-42b6-99ea-8f3080dc409a-kube-api-access-9sdrh\") pod \"dns-operator-controller-manager-864dd78c4c-mbfbx\" (UID: \"cd73322e-f25f-42b6-99ea-8f3080dc409a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mbfbx" Jun 15 03:17:58.013321 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:58.013230 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mbfbx" Jun 15 03:17:58.146861 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:58.146836 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mbfbx"] Jun 15 03:17:58.149074 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:17:58.149039 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd73322e_f25f_42b6_99ea_8f3080dc409a.slice/crio-37243e5dadd34b1bede1c69cf99746433d246d4b4dbaee4eb4af97f60249785a WatchSource:0}: Error finding container 37243e5dadd34b1bede1c69cf99746433d246d4b4dbaee4eb4af97f60249785a: Status 404 returned error can't find the container with id 37243e5dadd34b1bede1c69cf99746433d246d4b4dbaee4eb4af97f60249785a Jun 15 03:17:58.614906 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:17:58.614872 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mbfbx" event={"ID":"cd73322e-f25f-42b6-99ea-8f3080dc409a","Type":"ContainerStarted","Data":"37243e5dadd34b1bede1c69cf99746433d246d4b4dbaee4eb4af97f60249785a"} Jun 15 03:18:00.098602 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:00.098561 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh"] Jun 15 03:18:00.102954 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:00.102927 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" Jun 15 03:18:00.105721 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:00.105697 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hc9hj\"" Jun 15 03:18:00.111168 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:00.111141 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh"] Jun 15 03:18:00.138788 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:00.138743 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wngfm\" (UniqueName: \"kubernetes.io/projected/d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01-kube-api-access-wngfm\") pod \"limitador-operator-controller-manager-859d7b78bd-rxjnh\" (UID: \"d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" Jun 15 03:18:00.239392 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:00.239363 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wngfm\" (UniqueName: \"kubernetes.io/projected/d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01-kube-api-access-wngfm\") pod \"limitador-operator-controller-manager-859d7b78bd-rxjnh\" (UID: \"d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" Jun 15 03:18:00.248066 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:00.248043 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wngfm\" (UniqueName: \"kubernetes.io/projected/d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01-kube-api-access-wngfm\") pod \"limitador-operator-controller-manager-859d7b78bd-rxjnh\" (UID: \"d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" Jun 15 03:18:00.417226 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:00.417142 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" Jun 15 03:18:00.560964 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:00.560938 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh"] Jun 15 03:18:00.563671 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:18:00.563632 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3a3d48b_4f05_4a4e_aa7a_16abe69c4d01.slice/crio-62e57e6a7c1ebfcc4daf8a5dbaee415504309e701d3fc40c6f737812f7baf1d9 WatchSource:0}: Error finding container 62e57e6a7c1ebfcc4daf8a5dbaee415504309e701d3fc40c6f737812f7baf1d9: Status 404 returned error can't find the container with id 62e57e6a7c1ebfcc4daf8a5dbaee415504309e701d3fc40c6f737812f7baf1d9 Jun 15 03:18:00.627867 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:00.627839 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" event={"ID":"d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01","Type":"ContainerStarted","Data":"62e57e6a7c1ebfcc4daf8a5dbaee415504309e701d3fc40c6f737812f7baf1d9"} Jun 15 03:18:00.629442 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:00.629414 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mbfbx" event={"ID":"cd73322e-f25f-42b6-99ea-8f3080dc409a","Type":"ContainerStarted","Data":"cea700ee7833db692e42c61943aeeb7652510819fa91e132493fe1726269dcf6"} Jun 15 03:18:00.629573 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:00.629502 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mbfbx" Jun 15 03:18:00.669968 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:00.669866 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mbfbx" podStartSLOduration=1.574119488 podStartE2EDuration="3.669850089s" totalImagesPullingTime="2.095730601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:18:00.668469497 +0000 UTC m=+558.825083706" watchObservedRunningTime="2026-06-15 03:18:00.669850089 +0000 UTC m=+558.826464296" Jun 15 03:18:03.649627 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:03.649584 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" event={"ID":"d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01","Type":"ContainerStarted","Data":"bb92f2f556f91636cfb28fc5efb893fba850b2be37187e14ab83b8d1f8b52954"} Jun 15 03:18:03.650037 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:03.649730 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" Jun 15 03:18:03.667747 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:03.667698 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" podStartSLOduration=1.55897373 podStartE2EDuration="3.667684912s" totalImagesPullingTime="2.108711182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:18:03.666992804 +0000 UTC m=+561.823607027" watchObservedRunningTime="2026-06-15 03:18:03.667684912 +0000 UTC m=+561.824299120" Jun 15 03:18:11.639617 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:11.639584 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mbfbx" Jun 15 03:18:14.657768 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:14.657738 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" Jun 15 03:18:16.879889 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:16.879856 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh"] Jun 15 03:18:16.880301 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:16.880101 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" podUID="d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01" containerName="manager" containerID="cri-o://bb92f2f556f91636cfb28fc5efb893fba850b2be37187e14ab83b8d1f8b52954" gracePeriod=2 Jun 15 03:18:16.893293 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:16.893264 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh"] Jun 15 03:18:16.913041 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:16.913012 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vkms9"] Jun 15 03:18:16.913474 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:16.913459 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01" containerName="manager" Jun 15 03:18:16.913528 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:16.913476 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01" containerName="manager" Jun 15 03:18:16.913573 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:16.913563 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01" containerName="manager" Jun 15 03:18:16.916809 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:16.916791 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vkms9" Jun 15 03:18:16.919581 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:18:16.919550 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rxjnh\" is forbidden: User \"system:node:ip-10-0-139-167.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-167.ec2.internal' and this object" podUID="d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" Jun 15 03:18:16.929956 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:16.929933 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vkms9"] Jun 15 03:18:16.984740 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:16.984701 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sztwc\" (UniqueName: \"kubernetes.io/projected/240a7780-5ace-4f8d-a20c-f82f767439b3-kube-api-access-sztwc\") pod \"limitador-operator-controller-manager-859d7b78bd-vkms9\" (UID: \"240a7780-5ace-4f8d-a20c-f82f767439b3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vkms9" Jun 15 03:18:17.085687 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.085654 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sztwc\" (UniqueName: \"kubernetes.io/projected/240a7780-5ace-4f8d-a20c-f82f767439b3-kube-api-access-sztwc\") pod \"limitador-operator-controller-manager-859d7b78bd-vkms9\" (UID: \"240a7780-5ace-4f8d-a20c-f82f767439b3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vkms9" Jun 15 03:18:17.095106 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.095052 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sztwc\" (UniqueName: \"kubernetes.io/projected/240a7780-5ace-4f8d-a20c-f82f767439b3-kube-api-access-sztwc\") pod \"limitador-operator-controller-manager-859d7b78bd-vkms9\" (UID: \"240a7780-5ace-4f8d-a20c-f82f767439b3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vkms9" Jun 15 03:18:17.125667 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.125645 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" Jun 15 03:18:17.128282 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:18:17.128252 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rxjnh\" is forbidden: User \"system:node:ip-10-0-139-167.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-167.ec2.internal' and this object" podUID="d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" Jun 15 03:18:17.186697 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.186624 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wngfm\" (UniqueName: \"kubernetes.io/projected/d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01-kube-api-access-wngfm\") pod \"d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01\" (UID: \"d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01\") " Jun 15 03:18:17.188544 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.188520 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01-kube-api-access-wngfm" (OuterVolumeSpecName: "kube-api-access-wngfm") pod "d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01" (UID: "d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01"). InnerVolumeSpecName "kube-api-access-wngfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:18:17.260603 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.260558 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vkms9" Jun 15 03:18:17.287780 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.287752 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wngfm\" (UniqueName: \"kubernetes.io/projected/d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01-kube-api-access-wngfm\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:18:17.414783 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.414757 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vkms9"] Jun 15 03:18:17.417704 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:18:17.417677 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod240a7780_5ace_4f8d_a20c_f82f767439b3.slice/crio-0ca25faed99e606c908c70e489352454fecc979ef0fc5be902cd06410dbfb33c WatchSource:0}: Error finding container 0ca25faed99e606c908c70e489352454fecc979ef0fc5be902cd06410dbfb33c: Status 404 returned error can't find the container with id 0ca25faed99e606c908c70e489352454fecc979ef0fc5be902cd06410dbfb33c Jun 15 03:18:17.735914 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.735812 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vkms9" event={"ID":"240a7780-5ace-4f8d-a20c-f82f767439b3","Type":"ContainerStarted","Data":"0b8a4d7c7c1490a54cf5ee06e54330b5ea2d129466e4b092f5f1ac17e75c94a2"} Jun 15 03:18:17.735914 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.735868 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vkms9" event={"ID":"240a7780-5ace-4f8d-a20c-f82f767439b3","Type":"ContainerStarted","Data":"0ca25faed99e606c908c70e489352454fecc979ef0fc5be902cd06410dbfb33c"} Jun 15 03:18:17.735914 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.735899 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vkms9" Jun 15 03:18:17.737292 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.737260 2560 generic.go:356] "Generic (PLEG): container finished" podID="d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01" containerID="bb92f2f556f91636cfb28fc5efb893fba850b2be37187e14ab83b8d1f8b52954" exitCode=0 Jun 15 03:18:17.737510 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.737305 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" Jun 15 03:18:17.737510 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.737314 2560 scope.go:117] "RemoveContainer" containerID="bb92f2f556f91636cfb28fc5efb893fba850b2be37187e14ab83b8d1f8b52954" Jun 15 03:18:17.738356 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:18:17.738324 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rxjnh\" is forbidden: User \"system:node:ip-10-0-139-167.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-167.ec2.internal' and this object" podUID="d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" Jun 15 03:18:17.748996 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.748976 2560 scope.go:117] "RemoveContainer" containerID="bb92f2f556f91636cfb28fc5efb893fba850b2be37187e14ab83b8d1f8b52954" Jun 15 03:18:17.749309 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:18:17.749285 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb92f2f556f91636cfb28fc5efb893fba850b2be37187e14ab83b8d1f8b52954\": container with ID starting with bb92f2f556f91636cfb28fc5efb893fba850b2be37187e14ab83b8d1f8b52954 not found: ID does not exist" containerID="bb92f2f556f91636cfb28fc5efb893fba850b2be37187e14ab83b8d1f8b52954" Jun 15 03:18:17.749392 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.749320 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb92f2f556f91636cfb28fc5efb893fba850b2be37187e14ab83b8d1f8b52954"} err="failed to get container status \"bb92f2f556f91636cfb28fc5efb893fba850b2be37187e14ab83b8d1f8b52954\": rpc error: code = NotFound desc = could not find container \"bb92f2f556f91636cfb28fc5efb893fba850b2be37187e14ab83b8d1f8b52954\": container with ID starting with bb92f2f556f91636cfb28fc5efb893fba850b2be37187e14ab83b8d1f8b52954 not found: ID does not exist" Jun 15 03:18:17.758550 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:18:17.758525 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rxjnh\" is forbidden: User \"system:node:ip-10-0-139-167.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-167.ec2.internal' and this object" podUID="d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rxjnh" Jun 15 03:18:17.759471 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:17.759434 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vkms9" podStartSLOduration=1.7594218289999999 podStartE2EDuration="1.759421829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:18:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:18:17.756362287 +0000 UTC m=+575.912976495" watchObservedRunningTime="2026-06-15 03:18:17.759421829 +0000 UTC m=+575.916036038" Jun 15 03:18:18.268499 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:18.268466 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01" path="/var/lib/kubelet/pods/d3a3d48b-4f05-4a4e-aa7a-16abe69c4d01/volumes" Jun 15 03:18:28.746824 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:28.746793 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vkms9" Jun 15 03:18:42.255662 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:42.255631 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:18:42.257215 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:18:42.257196 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:19:00.725333 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:00.725249 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nmfvw"] Jun 15 03:19:00.727559 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:00.727544 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" Jun 15 03:19:00.730053 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:00.730024 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bkprt\"" Jun 15 03:19:00.730199 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:00.730036 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 15 03:19:00.736318 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:00.736293 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nmfvw"] Jun 15 03:19:00.827657 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:00.827625 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nmfvw"] Jun 15 03:19:00.872546 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:00.872515 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnwn4\" (UniqueName: \"kubernetes.io/projected/24f427b3-5343-4280-8e24-85651e927924-kube-api-access-gnwn4\") pod \"limitador-limitador-64c94fd64f-nmfvw\" (UID: \"24f427b3-5343-4280-8e24-85651e927924\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" Jun 15 03:19:00.872693 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:00.872629 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/24f427b3-5343-4280-8e24-85651e927924-config-file\") pod \"limitador-limitador-64c94fd64f-nmfvw\" (UID: \"24f427b3-5343-4280-8e24-85651e927924\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" Jun 15 03:19:00.973728 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:00.973692 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/24f427b3-5343-4280-8e24-85651e927924-config-file\") pod \"limitador-limitador-64c94fd64f-nmfvw\" (UID: \"24f427b3-5343-4280-8e24-85651e927924\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" Jun 15 03:19:00.973907 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:00.973756 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnwn4\" (UniqueName: \"kubernetes.io/projected/24f427b3-5343-4280-8e24-85651e927924-kube-api-access-gnwn4\") pod \"limitador-limitador-64c94fd64f-nmfvw\" (UID: \"24f427b3-5343-4280-8e24-85651e927924\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" Jun 15 03:19:00.974328 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:00.974308 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/24f427b3-5343-4280-8e24-85651e927924-config-file\") pod \"limitador-limitador-64c94fd64f-nmfvw\" (UID: \"24f427b3-5343-4280-8e24-85651e927924\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" Jun 15 03:19:00.982647 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:00.982588 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnwn4\" (UniqueName: \"kubernetes.io/projected/24f427b3-5343-4280-8e24-85651e927924-kube-api-access-gnwn4\") pod \"limitador-limitador-64c94fd64f-nmfvw\" (UID: \"24f427b3-5343-4280-8e24-85651e927924\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" Jun 15 03:19:01.038884 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:01.038853 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" Jun 15 03:19:01.177052 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:01.177023 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nmfvw"] Jun 15 03:19:01.178135 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:19:01.178112 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24f427b3_5343_4280_8e24_85651e927924.slice/crio-4a8ee4a374daf84cd6ead369d08abd248ffb1fe07866dd4dd5d16548e5836c6b WatchSource:0}: Error finding container 4a8ee4a374daf84cd6ead369d08abd248ffb1fe07866dd4dd5d16548e5836c6b: Status 404 returned error can't find the container with id 4a8ee4a374daf84cd6ead369d08abd248ffb1fe07866dd4dd5d16548e5836c6b Jun 15 03:19:01.998898 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:01.998856 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" event={"ID":"24f427b3-5343-4280-8e24-85651e927924","Type":"ContainerStarted","Data":"4a8ee4a374daf84cd6ead369d08abd248ffb1fe07866dd4dd5d16548e5836c6b"} Jun 15 03:19:04.012352 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:04.012320 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" event={"ID":"24f427b3-5343-4280-8e24-85651e927924","Type":"ContainerStarted","Data":"e218b048da23ea7ec8c7277bb6a9897449263918268335c559bc93b902153748"} Jun 15 03:19:04.012756 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:04.012435 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" Jun 15 03:19:04.029801 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:04.029744 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" podStartSLOduration=1.346700082 podStartE2EDuration="4.029726148s" totalImagesPullingTime="2.683026066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:19:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:19:04.028613182 +0000 UTC m=+622.185227402" watchObservedRunningTime="2026-06-15 03:19:04.029726148 +0000 UTC m=+622.186340355" Jun 15 03:19:15.019730 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:15.019697 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" Jun 15 03:19:15.492672 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:15.492626 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nmfvw"] Jun 15 03:19:15.492921 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:15.492890 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" podUID="24f427b3-5343-4280-8e24-85651e927924" containerName="limitador" containerID="cri-o://e218b048da23ea7ec8c7277bb6a9897449263918268335c559bc93b902153748" gracePeriod=30 Jun 15 03:19:16.043935 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.043913 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" Jun 15 03:19:16.082785 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.082702 2560 generic.go:356] "Generic (PLEG): container finished" podID="24f427b3-5343-4280-8e24-85651e927924" containerID="e218b048da23ea7ec8c7277bb6a9897449263918268335c559bc93b902153748" exitCode=0 Jun 15 03:19:16.082785 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.082744 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" Jun 15 03:19:16.082974 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.082790 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" event={"ID":"24f427b3-5343-4280-8e24-85651e927924","Type":"ContainerDied","Data":"e218b048da23ea7ec8c7277bb6a9897449263918268335c559bc93b902153748"} Jun 15 03:19:16.082974 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.082834 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmfvw" event={"ID":"24f427b3-5343-4280-8e24-85651e927924","Type":"ContainerDied","Data":"4a8ee4a374daf84cd6ead369d08abd248ffb1fe07866dd4dd5d16548e5836c6b"} Jun 15 03:19:16.082974 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.082856 2560 scope.go:117] "RemoveContainer" containerID="e218b048da23ea7ec8c7277bb6a9897449263918268335c559bc93b902153748" Jun 15 03:19:16.091683 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.091655 2560 scope.go:117] "RemoveContainer" containerID="e218b048da23ea7ec8c7277bb6a9897449263918268335c559bc93b902153748" Jun 15 03:19:16.091943 ip-10-0-139-167 kubenswrapper[2560]: E0615 03:19:16.091916 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e218b048da23ea7ec8c7277bb6a9897449263918268335c559bc93b902153748\": container with ID starting with e218b048da23ea7ec8c7277bb6a9897449263918268335c559bc93b902153748 not found: ID does not exist" containerID="e218b048da23ea7ec8c7277bb6a9897449263918268335c559bc93b902153748" Jun 15 03:19:16.091990 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.091945 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e218b048da23ea7ec8c7277bb6a9897449263918268335c559bc93b902153748"} err="failed to get container status \"e218b048da23ea7ec8c7277bb6a9897449263918268335c559bc93b902153748\": rpc error: code = NotFound desc = could not find container \"e218b048da23ea7ec8c7277bb6a9897449263918268335c559bc93b902153748\": container with ID starting with e218b048da23ea7ec8c7277bb6a9897449263918268335c559bc93b902153748 not found: ID does not exist" Jun 15 03:19:16.214910 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.214885 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnwn4\" (UniqueName: \"kubernetes.io/projected/24f427b3-5343-4280-8e24-85651e927924-kube-api-access-gnwn4\") pod \"24f427b3-5343-4280-8e24-85651e927924\" (UID: \"24f427b3-5343-4280-8e24-85651e927924\") " Jun 15 03:19:16.215072 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.214926 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/24f427b3-5343-4280-8e24-85651e927924-config-file\") pod \"24f427b3-5343-4280-8e24-85651e927924\" (UID: \"24f427b3-5343-4280-8e24-85651e927924\") " Jun 15 03:19:16.215325 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.215303 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24f427b3-5343-4280-8e24-85651e927924-config-file" (OuterVolumeSpecName: "config-file") pod "24f427b3-5343-4280-8e24-85651e927924" (UID: "24f427b3-5343-4280-8e24-85651e927924"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:19:16.217032 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.217010 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24f427b3-5343-4280-8e24-85651e927924-kube-api-access-gnwn4" (OuterVolumeSpecName: "kube-api-access-gnwn4") pod "24f427b3-5343-4280-8e24-85651e927924" (UID: "24f427b3-5343-4280-8e24-85651e927924"). InnerVolumeSpecName "kube-api-access-gnwn4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:19:16.315499 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.315471 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnwn4\" (UniqueName: \"kubernetes.io/projected/24f427b3-5343-4280-8e24-85651e927924-kube-api-access-gnwn4\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:19:16.315499 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.315495 2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/24f427b3-5343-4280-8e24-85651e927924-config-file\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:19:16.399987 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.399905 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nmfvw"] Jun 15 03:19:16.402208 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:16.402181 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nmfvw"] Jun 15 03:19:18.268250 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:18.268215 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24f427b3-5343-4280-8e24-85651e927924" path="/var/lib/kubelet/pods/24f427b3-5343-4280-8e24-85651e927924/volumes" Jun 15 03:19:19.032999 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.032962 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-kgpvn"] Jun 15 03:19:19.033558 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.033542 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24f427b3-5343-4280-8e24-85651e927924" containerName="limitador" Jun 15 03:19:19.033606 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.033562 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="24f427b3-5343-4280-8e24-85651e927924" containerName="limitador" Jun 15 03:19:19.033687 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.033676 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="24f427b3-5343-4280-8e24-85651e927924" containerName="limitador" Jun 15 03:19:19.037019 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.037000 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-kgpvn" Jun 15 03:19:19.040001 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.039975 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 15 03:19:19.042297 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.042274 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-kgpvn"] Jun 15 03:19:19.139897 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.139854 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f6f9f083-598d-424a-88d3-875c3d324f16-data\") pod \"postgres-69fb4594cf-kgpvn\" (UID: \"f6f9f083-598d-424a-88d3-875c3d324f16\") " pod="opendatahub/postgres-69fb4594cf-kgpvn" Jun 15 03:19:19.140068 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.139962 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsg8t\" (UniqueName: \"kubernetes.io/projected/f6f9f083-598d-424a-88d3-875c3d324f16-kube-api-access-nsg8t\") pod \"postgres-69fb4594cf-kgpvn\" (UID: \"f6f9f083-598d-424a-88d3-875c3d324f16\") " pod="opendatahub/postgres-69fb4594cf-kgpvn" Jun 15 03:19:19.241056 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.241005 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f6f9f083-598d-424a-88d3-875c3d324f16-data\") pod \"postgres-69fb4594cf-kgpvn\" (UID: \"f6f9f083-598d-424a-88d3-875c3d324f16\") " pod="opendatahub/postgres-69fb4594cf-kgpvn" Jun 15 03:19:19.241250 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.241143 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsg8t\" (UniqueName: \"kubernetes.io/projected/f6f9f083-598d-424a-88d3-875c3d324f16-kube-api-access-nsg8t\") pod \"postgres-69fb4594cf-kgpvn\" (UID: \"f6f9f083-598d-424a-88d3-875c3d324f16\") " pod="opendatahub/postgres-69fb4594cf-kgpvn" Jun 15 03:19:19.241441 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.241421 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f6f9f083-598d-424a-88d3-875c3d324f16-data\") pod \"postgres-69fb4594cf-kgpvn\" (UID: \"f6f9f083-598d-424a-88d3-875c3d324f16\") " pod="opendatahub/postgres-69fb4594cf-kgpvn" Jun 15 03:19:19.250171 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.250137 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsg8t\" (UniqueName: \"kubernetes.io/projected/f6f9f083-598d-424a-88d3-875c3d324f16-kube-api-access-nsg8t\") pod \"postgres-69fb4594cf-kgpvn\" (UID: \"f6f9f083-598d-424a-88d3-875c3d324f16\") " pod="opendatahub/postgres-69fb4594cf-kgpvn" Jun 15 03:19:19.348547 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.348454 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-kgpvn" Jun 15 03:19:19.478016 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.477987 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-kgpvn"] Jun 15 03:19:19.478716 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:19:19.478694 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f9f083_598d_424a_88d3_875c3d324f16.slice/crio-90016659de8711fd8500014605f00ddc2ae6fee0b931aff994b6e663eec9076c WatchSource:0}: Error finding container 90016659de8711fd8500014605f00ddc2ae6fee0b931aff994b6e663eec9076c: Status 404 returned error can't find the container with id 90016659de8711fd8500014605f00ddc2ae6fee0b931aff994b6e663eec9076c Jun 15 03:19:19.480119 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:19.480077 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 03:19:20.106593 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:20.106548 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-kgpvn" event={"ID":"f6f9f083-598d-424a-88d3-875c3d324f16","Type":"ContainerStarted","Data":"90016659de8711fd8500014605f00ddc2ae6fee0b931aff994b6e663eec9076c"} Jun 15 03:19:22.120795 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:22.120713 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-kgpvn" event={"ID":"f6f9f083-598d-424a-88d3-875c3d324f16","Type":"ContainerStarted","Data":"ea95e92d0a46bc40aca2dd9877f32ce0c12b996e3af5465a924d01b20e931322"} Jun 15 03:19:22.120795 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:22.120790 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-kgpvn" Jun 15 03:19:22.139074 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:22.139016 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-kgpvn" podStartSLOduration=0.759386408 podStartE2EDuration="3.138999466s" totalImagesPullingTime="2.379613058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:19:22.136271646 +0000 UTC m=+640.292885865" watchObservedRunningTime="2026-06-15 03:19:22.138999466 +0000 UTC m=+640.295613673" Jun 15 03:19:28.159623 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:28.159592 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-kgpvn" Jun 15 03:19:34.180557 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.180525 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp"] Jun 15 03:19:34.187825 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.187804 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" Jun 15 03:19:34.190545 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.190514 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9h8zm\"" Jun 15 03:19:34.190545 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.190530 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 03:19:34.191791 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.191771 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 03:19:34.192935 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.192913 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp"] Jun 15 03:19:34.276409 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.276378 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56414ce2-eb73-4bf7-b3df-25c95316d1b3-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp\" (UID: \"56414ce2-eb73-4bf7-b3df-25c95316d1b3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" Jun 15 03:19:34.276601 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.276426 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56414ce2-eb73-4bf7-b3df-25c95316d1b3-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp\" (UID: \"56414ce2-eb73-4bf7-b3df-25c95316d1b3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" Jun 15 03:19:34.276601 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.276452 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss8gq\" (UniqueName: \"kubernetes.io/projected/56414ce2-eb73-4bf7-b3df-25c95316d1b3-kube-api-access-ss8gq\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp\" (UID: \"56414ce2-eb73-4bf7-b3df-25c95316d1b3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" Jun 15 03:19:34.377621 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.377583 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56414ce2-eb73-4bf7-b3df-25c95316d1b3-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp\" (UID: \"56414ce2-eb73-4bf7-b3df-25c95316d1b3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" Jun 15 03:19:34.377798 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.377631 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56414ce2-eb73-4bf7-b3df-25c95316d1b3-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp\" (UID: \"56414ce2-eb73-4bf7-b3df-25c95316d1b3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" Jun 15 03:19:34.377798 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.377737 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss8gq\" (UniqueName: \"kubernetes.io/projected/56414ce2-eb73-4bf7-b3df-25c95316d1b3-kube-api-access-ss8gq\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp\" (UID: \"56414ce2-eb73-4bf7-b3df-25c95316d1b3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" Jun 15 03:19:34.377980 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.377963 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56414ce2-eb73-4bf7-b3df-25c95316d1b3-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp\" (UID: \"56414ce2-eb73-4bf7-b3df-25c95316d1b3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" Jun 15 03:19:34.378044 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.378028 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56414ce2-eb73-4bf7-b3df-25c95316d1b3-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp\" (UID: \"56414ce2-eb73-4bf7-b3df-25c95316d1b3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" Jun 15 03:19:34.386557 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.386531 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss8gq\" (UniqueName: \"kubernetes.io/projected/56414ce2-eb73-4bf7-b3df-25c95316d1b3-kube-api-access-ss8gq\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp\" (UID: \"56414ce2-eb73-4bf7-b3df-25c95316d1b3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" Jun 15 03:19:34.498884 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.498800 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" Jun 15 03:19:34.632845 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:34.632818 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp"] Jun 15 03:19:34.634441 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:19:34.634412 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56414ce2_eb73_4bf7_b3df_25c95316d1b3.slice/crio-1866379f264bd11a1c4c5bca964341c3acb5bccf7fa801b5bad031753ccc3aec WatchSource:0}: Error finding container 1866379f264bd11a1c4c5bca964341c3acb5bccf7fa801b5bad031753ccc3aec: Status 404 returned error can't find the container with id 1866379f264bd11a1c4c5bca964341c3acb5bccf7fa801b5bad031753ccc3aec Jun 15 03:19:35.201942 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:35.201908 2560 generic.go:356] "Generic (PLEG): container finished" podID="56414ce2-eb73-4bf7-b3df-25c95316d1b3" containerID="2e7798b2f350e6932613e9f05db9076825a5717755239a137f0dbc45a839f6d2" exitCode=0 Jun 15 03:19:35.202416 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:35.201970 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" event={"ID":"56414ce2-eb73-4bf7-b3df-25c95316d1b3","Type":"ContainerDied","Data":"2e7798b2f350e6932613e9f05db9076825a5717755239a137f0dbc45a839f6d2"} Jun 15 03:19:35.202416 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:35.202004 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" event={"ID":"56414ce2-eb73-4bf7-b3df-25c95316d1b3","Type":"ContainerStarted","Data":"1866379f264bd11a1c4c5bca964341c3acb5bccf7fa801b5bad031753ccc3aec"} Jun 15 03:19:36.209606 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:36.209560 2560 generic.go:356] "Generic (PLEG): container finished" podID="56414ce2-eb73-4bf7-b3df-25c95316d1b3" containerID="b346d05484fb97c535cc045b3b5648f0cc05b5551956e70d5ed93b7d85f0c7ac" exitCode=0 Jun 15 03:19:36.209975 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:36.209628 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" event={"ID":"56414ce2-eb73-4bf7-b3df-25c95316d1b3","Type":"ContainerDied","Data":"b346d05484fb97c535cc045b3b5648f0cc05b5551956e70d5ed93b7d85f0c7ac"} Jun 15 03:19:37.217948 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:37.217910 2560 generic.go:356] "Generic (PLEG): container finished" podID="56414ce2-eb73-4bf7-b3df-25c95316d1b3" containerID="52441ebd57a55ed23480ac1d273757c80ed917fb4bd53b63ed301e9cd8d03b23" exitCode=0 Jun 15 03:19:37.218362 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:37.217993 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" event={"ID":"56414ce2-eb73-4bf7-b3df-25c95316d1b3","Type":"ContainerDied","Data":"52441ebd57a55ed23480ac1d273757c80ed917fb4bd53b63ed301e9cd8d03b23"} Jun 15 03:19:38.356824 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:38.356799 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" Jun 15 03:19:38.415060 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:38.415026 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56414ce2-eb73-4bf7-b3df-25c95316d1b3-util\") pod \"56414ce2-eb73-4bf7-b3df-25c95316d1b3\" (UID: \"56414ce2-eb73-4bf7-b3df-25c95316d1b3\") " Jun 15 03:19:38.415216 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:38.415131 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56414ce2-eb73-4bf7-b3df-25c95316d1b3-bundle\") pod \"56414ce2-eb73-4bf7-b3df-25c95316d1b3\" (UID: \"56414ce2-eb73-4bf7-b3df-25c95316d1b3\") " Jun 15 03:19:38.415216 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:38.415175 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss8gq\" (UniqueName: \"kubernetes.io/projected/56414ce2-eb73-4bf7-b3df-25c95316d1b3-kube-api-access-ss8gq\") pod \"56414ce2-eb73-4bf7-b3df-25c95316d1b3\" (UID: \"56414ce2-eb73-4bf7-b3df-25c95316d1b3\") " Jun 15 03:19:38.415604 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:38.415574 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56414ce2-eb73-4bf7-b3df-25c95316d1b3-bundle" (OuterVolumeSpecName: "bundle") pod "56414ce2-eb73-4bf7-b3df-25c95316d1b3" (UID: "56414ce2-eb73-4bf7-b3df-25c95316d1b3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:19:38.417310 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:38.417280 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56414ce2-eb73-4bf7-b3df-25c95316d1b3-kube-api-access-ss8gq" (OuterVolumeSpecName: "kube-api-access-ss8gq") pod "56414ce2-eb73-4bf7-b3df-25c95316d1b3" (UID: "56414ce2-eb73-4bf7-b3df-25c95316d1b3"). InnerVolumeSpecName "kube-api-access-ss8gq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:19:38.420460 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:38.420422 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56414ce2-eb73-4bf7-b3df-25c95316d1b3-util" (OuterVolumeSpecName: "util") pod "56414ce2-eb73-4bf7-b3df-25c95316d1b3" (UID: "56414ce2-eb73-4bf7-b3df-25c95316d1b3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:19:38.516046 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:38.515942 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56414ce2-eb73-4bf7-b3df-25c95316d1b3-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:19:38.516046 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:38.515981 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss8gq\" (UniqueName: \"kubernetes.io/projected/56414ce2-eb73-4bf7-b3df-25c95316d1b3-kube-api-access-ss8gq\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:19:38.516046 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:38.515994 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56414ce2-eb73-4bf7-b3df-25c95316d1b3-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:19:39.232108 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:39.232062 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" event={"ID":"56414ce2-eb73-4bf7-b3df-25c95316d1b3","Type":"ContainerDied","Data":"1866379f264bd11a1c4c5bca964341c3acb5bccf7fa801b5bad031753ccc3aec"} Jun 15 03:19:39.232301 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:39.232119 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485czbp" Jun 15 03:19:39.232301 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:39.232127 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1866379f264bd11a1c4c5bca964341c3acb5bccf7fa801b5bad031753ccc3aec" Jun 15 03:19:43.047283 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.047249 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-5t58n"] Jun 15 03:19:43.047650 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.047630 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56414ce2-eb73-4bf7-b3df-25c95316d1b3" containerName="extract" Jun 15 03:19:43.047650 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.047642 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="56414ce2-eb73-4bf7-b3df-25c95316d1b3" containerName="extract" Jun 15 03:19:43.047717 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.047654 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56414ce2-eb73-4bf7-b3df-25c95316d1b3" containerName="util" Jun 15 03:19:43.047717 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.047660 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="56414ce2-eb73-4bf7-b3df-25c95316d1b3" containerName="util" Jun 15 03:19:43.047717 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.047671 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56414ce2-eb73-4bf7-b3df-25c95316d1b3" containerName="pull" Jun 15 03:19:43.047717 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.047677 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="56414ce2-eb73-4bf7-b3df-25c95316d1b3" containerName="pull" Jun 15 03:19:43.047837 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.047736 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="56414ce2-eb73-4bf7-b3df-25c95316d1b3" containerName="extract" Jun 15 03:19:43.050279 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.050258 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-5t58n" Jun 15 03:19:43.053258 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.053236 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 15 03:19:43.053362 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.053278 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 15 03:19:43.054426 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.054410 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"keycloak-operator-dockercfg-4msc4\"" Jun 15 03:19:43.058570 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.058548 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-5t58n"] Jun 15 03:19:43.156640 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.156601 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv557\" (UniqueName: \"kubernetes.io/projected/ce485a03-94b8-4f08-b4ba-a2f43a3a0df0-kube-api-access-jv557\") pod \"keycloak-operator-d67dbbb48-5t58n\" (UID: \"ce485a03-94b8-4f08-b4ba-a2f43a3a0df0\") " pod="keycloak-system/keycloak-operator-d67dbbb48-5t58n" Jun 15 03:19:43.257628 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.257601 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv557\" (UniqueName: \"kubernetes.io/projected/ce485a03-94b8-4f08-b4ba-a2f43a3a0df0-kube-api-access-jv557\") pod \"keycloak-operator-d67dbbb48-5t58n\" (UID: \"ce485a03-94b8-4f08-b4ba-a2f43a3a0df0\") " pod="keycloak-system/keycloak-operator-d67dbbb48-5t58n" Jun 15 03:19:43.266536 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.266510 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv557\" (UniqueName: \"kubernetes.io/projected/ce485a03-94b8-4f08-b4ba-a2f43a3a0df0-kube-api-access-jv557\") pod \"keycloak-operator-d67dbbb48-5t58n\" (UID: \"ce485a03-94b8-4f08-b4ba-a2f43a3a0df0\") " pod="keycloak-system/keycloak-operator-d67dbbb48-5t58n" Jun 15 03:19:43.361923 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.361856 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-5t58n" Jun 15 03:19:43.496046 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:43.496019 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-5t58n"] Jun 15 03:19:43.497283 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:19:43.497257 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce485a03_94b8_4f08_b4ba_a2f43a3a0df0.slice/crio-13b9c34fe94dc644a1c451564fb680d7b7ddc437825db017b3a713c3fe580566 WatchSource:0}: Error finding container 13b9c34fe94dc644a1c451564fb680d7b7ddc437825db017b3a713c3fe580566: Status 404 returned error can't find the container with id 13b9c34fe94dc644a1c451564fb680d7b7ddc437825db017b3a713c3fe580566 Jun 15 03:19:44.286839 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:44.286796 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-5t58n" event={"ID":"ce485a03-94b8-4f08-b4ba-a2f43a3a0df0","Type":"ContainerStarted","Data":"13b9c34fe94dc644a1c451564fb680d7b7ddc437825db017b3a713c3fe580566"} Jun 15 03:19:50.317620 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:50.317578 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-5t58n" event={"ID":"ce485a03-94b8-4f08-b4ba-a2f43a3a0df0","Type":"ContainerStarted","Data":"ce63c2d38747aad77b24f24de28146a4806be1cd05b53bf1cc7d4cf7b9cc47fd"} Jun 15 03:19:50.335937 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:19:50.335890 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/keycloak-operator-d67dbbb48-5t58n" podStartSLOduration=0.713478545 podStartE2EDuration="7.33587444s" totalImagesPullingTime="6.622395895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:19:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:19:50.333875675 +0000 UTC m=+668.490489884" watchObservedRunningTime="2026-06-15 03:19:50.33587444 +0000 UTC m=+668.492488648" Jun 15 03:22:21.020015 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.019973 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl"] Jun 15 03:22:21.022839 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.022815 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.027096 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.027057 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 15 03:22:21.027204 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.027134 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 15 03:22:21.027204 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.027145 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nt4nn\"" Jun 15 03:22:21.027316 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.027245 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 15 03:22:21.032356 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.032336 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl"] Jun 15 03:22:21.168666 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.168584 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b5740581-07cd-4d27-8643-e03cd7f31a5a-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.168666 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.168630 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5740581-07cd-4d27-8643-e03cd7f31a5a-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.168666 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.168660 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5740581-07cd-4d27-8643-e03cd7f31a5a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.168900 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.168694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5740581-07cd-4d27-8643-e03cd7f31a5a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.168900 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.168711 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5740581-07cd-4d27-8643-e03cd7f31a5a-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.168900 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.168730 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mgtv\" (UniqueName: \"kubernetes.io/projected/b5740581-07cd-4d27-8643-e03cd7f31a5a-kube-api-access-2mgtv\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.269196 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.269166 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5740581-07cd-4d27-8643-e03cd7f31a5a-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.269404 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.269203 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mgtv\" (UniqueName: \"kubernetes.io/projected/b5740581-07cd-4d27-8643-e03cd7f31a5a-kube-api-access-2mgtv\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.269404 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.269286 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b5740581-07cd-4d27-8643-e03cd7f31a5a-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.269404 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.269328 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5740581-07cd-4d27-8643-e03cd7f31a5a-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.269404 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.269362 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5740581-07cd-4d27-8643-e03cd7f31a5a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.269404 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.269390 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5740581-07cd-4d27-8643-e03cd7f31a5a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.269726 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.269701 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b5740581-07cd-4d27-8643-e03cd7f31a5a-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.269726 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.269721 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5740581-07cd-4d27-8643-e03cd7f31a5a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.269840 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.269754 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5740581-07cd-4d27-8643-e03cd7f31a5a-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.271632 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.271613 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5740581-07cd-4d27-8643-e03cd7f31a5a-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.271869 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.271851 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5740581-07cd-4d27-8643-e03cd7f31a5a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.277622 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.277595 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mgtv\" (UniqueName: \"kubernetes.io/projected/b5740581-07cd-4d27-8643-e03cd7f31a5a-kube-api-access-2mgtv\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl\" (UID: \"b5740581-07cd-4d27-8643-e03cd7f31a5a\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.333197 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.333178 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:22:21.468883 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:21.468853 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl"] Jun 15 03:22:21.470162 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:22:21.470138 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5740581_07cd_4d27_8643_e03cd7f31a5a.slice/crio-64fb91cf950a987033825051a8c596702d3573b0f771ec276fa3c8bccecff0be WatchSource:0}: Error finding container 64fb91cf950a987033825051a8c596702d3573b0f771ec276fa3c8bccecff0be: Status 404 returned error can't find the container with id 64fb91cf950a987033825051a8c596702d3573b0f771ec276fa3c8bccecff0be Jun 15 03:22:22.212820 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:22.212759 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" event={"ID":"b5740581-07cd-4d27-8643-e03cd7f31a5a","Type":"ContainerStarted","Data":"64fb91cf950a987033825051a8c596702d3573b0f771ec276fa3c8bccecff0be"} Jun 15 03:22:25.233581 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:22:25.233546 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" event={"ID":"b5740581-07cd-4d27-8643-e03cd7f31a5a","Type":"ContainerStarted","Data":"67606598ef0b4f3845955193c8aeeb0927306306cc90b2fe2086471150762b1e"} Jun 15 03:23:06.482091 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:23:06.482044 2560 generic.go:356] "Generic (PLEG): container finished" podID="b5740581-07cd-4d27-8643-e03cd7f31a5a" containerID="67606598ef0b4f3845955193c8aeeb0927306306cc90b2fe2086471150762b1e" exitCode=0 Jun 15 03:23:06.482554 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:23:06.482120 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" event={"ID":"b5740581-07cd-4d27-8643-e03cd7f31a5a","Type":"ContainerDied","Data":"67606598ef0b4f3845955193c8aeeb0927306306cc90b2fe2086471150762b1e"} Jun 15 03:23:08.496582 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:23:08.496546 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" event={"ID":"b5740581-07cd-4d27-8643-e03cd7f31a5a","Type":"ContainerStarted","Data":"70f6e22556468a1b4d7c6c929f658765677b6a08cd43c42c96cc5d06a47b87d3"} Jun 15 03:23:08.517949 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:23:08.517901 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" podStartSLOduration=3.064631843 podStartE2EDuration="48.51788345s" totalImagesPullingTime="4.037496326s" totalInitContainerRuntime="41.415755281s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:22:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:23:08.514518804 +0000 UTC m=+866.671133035" watchObservedRunningTime="2026-06-15 03:23:08.51788345 +0000 UTC m=+866.674497703" Jun 15 03:23:11.334133 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:23:11.334078 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:23:11.334606 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:23:11.334177 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:23:11.349007 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:23:11.348984 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:23:11.526598 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:23:11.526570 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl" Jun 15 03:23:42.303162 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:23:42.303068 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:23:42.306784 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:23:42.306764 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:28:42.344214 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:28:42.344183 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:28:42.348706 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:28:42.348686 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:30:00.142823 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:00.142789 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29691570-4wm89"] Jun 15 03:30:00.146585 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:00.146565 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29691570-4wm89" Jun 15 03:30:00.149688 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:00.149667 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-lvlnl\"" Jun 15 03:30:00.164626 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:00.164597 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29691570-4wm89"] Jun 15 03:30:00.237027 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:00.236992 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z466g\" (UniqueName: \"kubernetes.io/projected/cce0e4ea-d1b5-489b-9976-fc997f7a17b2-kube-api-access-z466g\") pod \"maas-api-key-cleanup-29691570-4wm89\" (UID: \"cce0e4ea-d1b5-489b-9976-fc997f7a17b2\") " pod="opendatahub/maas-api-key-cleanup-29691570-4wm89" Jun 15 03:30:00.338022 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:00.337991 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z466g\" (UniqueName: \"kubernetes.io/projected/cce0e4ea-d1b5-489b-9976-fc997f7a17b2-kube-api-access-z466g\") pod \"maas-api-key-cleanup-29691570-4wm89\" (UID: \"cce0e4ea-d1b5-489b-9976-fc997f7a17b2\") " pod="opendatahub/maas-api-key-cleanup-29691570-4wm89" Jun 15 03:30:00.347225 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:00.347196 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z466g\" (UniqueName: \"kubernetes.io/projected/cce0e4ea-d1b5-489b-9976-fc997f7a17b2-kube-api-access-z466g\") pod \"maas-api-key-cleanup-29691570-4wm89\" (UID: \"cce0e4ea-d1b5-489b-9976-fc997f7a17b2\") " pod="opendatahub/maas-api-key-cleanup-29691570-4wm89" Jun 15 03:30:00.457513 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:00.457424 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29691570-4wm89" Jun 15 03:30:00.592373 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:00.592346 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29691570-4wm89"] Jun 15 03:30:00.594294 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:30:00.594267 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcce0e4ea_d1b5_489b_9976_fc997f7a17b2.slice/crio-129bef717c40018a44b8c8a766d6cec3a1d0a3820dadf433d8280277d787f4d4 WatchSource:0}: Error finding container 129bef717c40018a44b8c8a766d6cec3a1d0a3820dadf433d8280277d787f4d4: Status 404 returned error can't find the container with id 129bef717c40018a44b8c8a766d6cec3a1d0a3820dadf433d8280277d787f4d4 Jun 15 03:30:00.596062 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:00.596043 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 03:30:00.889927 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:00.889889 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29691570-4wm89" event={"ID":"cce0e4ea-d1b5-489b-9976-fc997f7a17b2","Type":"ContainerStarted","Data":"129bef717c40018a44b8c8a766d6cec3a1d0a3820dadf433d8280277d787f4d4"} Jun 15 03:30:15.980184 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:15.980154 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29691570-4wm89" event={"ID":"cce0e4ea-d1b5-489b-9976-fc997f7a17b2","Type":"ContainerStarted","Data":"8112fa39b1dcfdf24bf3c880167f666ae2aa1c06d5391d84ce5f66a937aadea6"} Jun 15 03:30:15.997963 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:15.997919 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29691570-4wm89" podStartSLOduration=0.972478879 podStartE2EDuration="15.997902925s" totalImagesPullingTime="15.025424046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:30:15.995314212 +0000 UTC m=+1294.151928443" watchObservedRunningTime="2026-06-15 03:30:15.997902925 +0000 UTC m=+1294.154517132" Jun 15 03:30:16.987578 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:16.987483 2560 generic.go:356] "Generic (PLEG): container finished" podID="cce0e4ea-d1b5-489b-9976-fc997f7a17b2" containerID="8112fa39b1dcfdf24bf3c880167f666ae2aa1c06d5391d84ce5f66a937aadea6" exitCode=0 Jun 15 03:30:16.987578 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:16.987534 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29691570-4wm89" event={"ID":"cce0e4ea-d1b5-489b-9976-fc997f7a17b2","Type":"ContainerDied","Data":"8112fa39b1dcfdf24bf3c880167f666ae2aa1c06d5391d84ce5f66a937aadea6"} Jun 15 03:30:18.119177 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:18.119154 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29691570-4wm89" Jun 15 03:30:18.302731 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:18.302706 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z466g\" (UniqueName: \"kubernetes.io/projected/cce0e4ea-d1b5-489b-9976-fc997f7a17b2-kube-api-access-z466g\") pod \"cce0e4ea-d1b5-489b-9976-fc997f7a17b2\" (UID: \"cce0e4ea-d1b5-489b-9976-fc997f7a17b2\") " Jun 15 03:30:18.304828 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:18.304805 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cce0e4ea-d1b5-489b-9976-fc997f7a17b2-kube-api-access-z466g" (OuterVolumeSpecName: "kube-api-access-z466g") pod "cce0e4ea-d1b5-489b-9976-fc997f7a17b2" (UID: "cce0e4ea-d1b5-489b-9976-fc997f7a17b2"). InnerVolumeSpecName "kube-api-access-z466g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:30:18.403651 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:18.403616 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z466g\" (UniqueName: \"kubernetes.io/projected/cce0e4ea-d1b5-489b-9976-fc997f7a17b2-kube-api-access-z466g\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jun 15 03:30:19.000420 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:19.000390 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29691570-4wm89" Jun 15 03:30:19.000589 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:19.000390 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29691570-4wm89" event={"ID":"cce0e4ea-d1b5-489b-9976-fc997f7a17b2","Type":"ContainerDied","Data":"129bef717c40018a44b8c8a766d6cec3a1d0a3820dadf433d8280277d787f4d4"} Jun 15 03:30:19.000589 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:30:19.000521 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="129bef717c40018a44b8c8a766d6cec3a1d0a3820dadf433d8280277d787f4d4" Jun 15 03:33:42.384553 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:33:42.384526 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:33:42.391424 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:33:42.391408 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:38:42.427009 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:38:42.426981 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:38:42.431979 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:38:42.431960 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:43:42.469157 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:43:42.469123 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:43:42.476827 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:43:42.476809 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:48:42.510857 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:48:42.510744 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:48:42.520395 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:48:42.520377 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:53:42.551171 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:53:42.551148 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:53:42.561831 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:53:42.561811 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vsz59_8f448e18-ba99-4279-b13a-b86003e4e2df/ovn-acl-logging/0.log" Jun 15 03:54:22.732268 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:22.732241 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-694cb6497d-qpx9v_7749efc3-a41d-4aac-a75d-f28b4f1b9ec0/nginx/0.log" Jun 15 03:54:22.835786 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:22.835750 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-dbmcd_9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc/manager/0.log" Jun 15 03:54:22.943275 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:22.943246 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-xmklr_3f22b40b-22cd-43b1-892b-f0fc87707156/manager/0.log" Jun 15 03:54:23.155606 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:23.155582 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29691570-4wm89_cce0e4ea-d1b5-489b-9976-fc997f7a17b2/cleanup/0.log" Jun 15 03:54:23.480674 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:23.480595 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-xsk7z_5570bf43-0431-4cd1-8c78-c02ff1ac57e2/server/0.log" Jun 15 03:54:23.594817 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:23.594790 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-jnphp_419a1975-d79b-4623-853b-bba16bc0e091/manager/0.log" Jun 15 03:54:23.942324 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:23.942300 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5799f64c8f-zd8s7_c12ee043-c8d5-4ad7-b955-bb125df4d5a0/copy-manifests/0.log" Jun 15 03:54:23.950300 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:23.950273 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5799f64c8f-zd8s7_c12ee043-c8d5-4ad7-b955-bb125df4d5a0/manager/0.log" Jun 15 03:54:24.056003 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:24.055980 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-kgpvn_f6f9f083-598d-424a-88d3-875c3d324f16/postgres/0.log" Jun 15 03:54:24.776075 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:24.776045 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk_b5ee597b-dd27-42c6-8351-5ddefd113f9f/util/0.log" Jun 15 03:54:24.782539 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:24.782519 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk_b5ee597b-dd27-42c6-8351-5ddefd113f9f/pull/0.log" Jun 15 03:54:24.788596 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:24.788579 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk_b5ee597b-dd27-42c6-8351-5ddefd113f9f/extract/0.log" Jun 15 03:54:24.893627 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:24.893603 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl_41832d93-7b55-4125-8f59-b5ea5a61fa73/pull/0.log" Jun 15 03:54:24.899703 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:24.899679 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl_41832d93-7b55-4125-8f59-b5ea5a61fa73/extract/0.log" Jun 15 03:54:24.905497 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:24.905478 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl_41832d93-7b55-4125-8f59-b5ea5a61fa73/util/0.log" Jun 15 03:54:25.010564 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:25.010541 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq_71d99a6d-64e4-406e-be19-7cb8f58fe719/extract/0.log" Jun 15 03:54:25.015610 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:25.015594 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq_71d99a6d-64e4-406e-be19-7cb8f58fe719/util/0.log" Jun 15 03:54:25.021302 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:25.021281 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq_71d99a6d-64e4-406e-be19-7cb8f58fe719/pull/0.log" Jun 15 03:54:25.125174 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:25.125155 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9_5bf52ff2-469e-4c84-ae4c-e18b0a33f139/util/0.log" Jun 15 03:54:25.131500 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:25.131484 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9_5bf52ff2-469e-4c84-ae4c-e18b0a33f139/pull/0.log" Jun 15 03:54:25.137169 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:25.137154 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9_5bf52ff2-469e-4c84-ae4c-e18b0a33f139/extract/0.log" Jun 15 03:54:25.461794 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:25.461715 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-mbfbx_cd73322e-f25f-42b6-99ea-8f3080dc409a/manager/0.log" Jun 15 03:54:26.016313 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:26.016284 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vkms9_240a7780-5ace-4f8d-a20c-f82f767439b3/manager/0.log" Jun 15 03:54:26.447367 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:26.447336 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-l7p2l_0429504b-413e-45fb-ae04-8345e31b1f31/discovery/0.log" Jun 15 03:54:26.547924 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:26.547896 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7579ff4b78-jgtz4_77f6c16f-a5eb-4963-abb0-db8b2ad4db30/kube-auth-proxy/0.log" Jun 15 03:54:27.090559 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:27.090527 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55f78b5b6d-xfjzj_1429058a-5113-4842-83d5-1f57f1fee24a/router/0.log" Jun 15 03:54:27.647930 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:27.647899 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl_b5740581-07cd-4d27-8643-e03cd7f31a5a/storage-initializer/0.log" Jun 15 03:54:27.654735 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:27.654683 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-66xpl_b5740581-07cd-4d27-8643-e03cd7f31a5a/main/0.log" Jun 15 03:54:35.187013 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:35.186977 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-66qb8_04ea9897-2b15-4a68-93cf-cad10b6c7245/global-pull-secret-syncer/0.log" Jun 15 03:54:35.338310 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:35.338281 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gc7b7_b1254732-a9f4-4953-80bc-4e01078878ac/konnectivity-agent/0.log" Jun 15 03:54:35.412893 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:35.412854 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-167.ec2.internal_3ccc0e83fda6561eb368da988cc50ea9/haproxy/0.log" Jun 15 03:54:38.143466 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:38.143436 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_servicemesh-operator3-57f65f65fb-84drr_fc2d3adb-1049-4977-9ae9-ec9285b4c40c/sail-operator/0.log" Jun 15 03:54:40.110476 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:40.110446 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk_b5ee597b-dd27-42c6-8351-5ddefd113f9f/extract/0.log" Jun 15 03:54:40.138362 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:40.138336 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk_b5ee597b-dd27-42c6-8351-5ddefd113f9f/util/0.log" Jun 15 03:54:40.159957 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:40.159935 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6dxk_b5ee597b-dd27-42c6-8351-5ddefd113f9f/pull/0.log" Jun 15 03:54:40.186818 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:40.186799 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl_41832d93-7b55-4125-8f59-b5ea5a61fa73/extract/0.log" Jun 15 03:54:40.208353 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:40.208333 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl_41832d93-7b55-4125-8f59-b5ea5a61fa73/util/0.log" Jun 15 03:54:40.234753 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:40.234731 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mx6hl_41832d93-7b55-4125-8f59-b5ea5a61fa73/pull/0.log" Jun 15 03:54:40.260815 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:40.260788 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq_71d99a6d-64e4-406e-be19-7cb8f58fe719/extract/0.log" Jun 15 03:54:40.281471 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:40.281444 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq_71d99a6d-64e4-406e-be19-7cb8f58fe719/util/0.log" Jun 15 03:54:40.303717 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:40.303670 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s2xzq_71d99a6d-64e4-406e-be19-7cb8f58fe719/pull/0.log" Jun 15 03:54:40.329669 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:40.329649 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9_5bf52ff2-469e-4c84-ae4c-e18b0a33f139/extract/0.log" Jun 15 03:54:40.350794 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:40.350774 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9_5bf52ff2-469e-4c84-ae4c-e18b0a33f139/util/0.log" Jun 15 03:54:40.379108 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:40.379005 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19kzt9_5bf52ff2-469e-4c84-ae4c-e18b0a33f139/pull/0.log" Jun 15 03:54:40.656781 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:40.656703 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-mbfbx_cd73322e-f25f-42b6-99ea-8f3080dc409a/manager/0.log" Jun 15 03:54:40.955919 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:40.955838 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vkms9_240a7780-5ace-4f8d-a20c-f82f767439b3/manager/0.log" Jun 15 03:54:42.615214 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:42.615182 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_54f1a04f-7091-4542-bc6b-e1c137c16710/alertmanager/0.log" Jun 15 03:54:42.635052 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:42.635030 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_54f1a04f-7091-4542-bc6b-e1c137c16710/config-reloader/0.log" Jun 15 03:54:42.662868 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:42.662842 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_54f1a04f-7091-4542-bc6b-e1c137c16710/kube-rbac-proxy-web/0.log" Jun 15 03:54:42.685053 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:42.685033 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_54f1a04f-7091-4542-bc6b-e1c137c16710/kube-rbac-proxy/0.log" Jun 15 03:54:42.708305 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:42.708285 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_54f1a04f-7091-4542-bc6b-e1c137c16710/kube-rbac-proxy-metric/0.log" Jun 15 03:54:42.728920 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:42.728896 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_54f1a04f-7091-4542-bc6b-e1c137c16710/prom-label-proxy/0.log" Jun 15 03:54:42.751887 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:42.751871 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_54f1a04f-7091-4542-bc6b-e1c137c16710/init-config-reloader/0.log" Jun 15 03:54:42.793484 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:42.793462 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-f9d7df769-9mvkb_cde73dc3-0cfa-4b1b-afe1-296bdf4e026c/cluster-monitoring-operator/0.log" Jun 15 03:54:42.949720 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:42.949640 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bsdj7_18a538b6-74bf-4c7a-a451-3dda824187cb/node-exporter/0.log" Jun 15 03:54:42.970804 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:42.970783 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bsdj7_18a538b6-74bf-4c7a-a451-3dda824187cb/kube-rbac-proxy/0.log" Jun 15 03:54:42.993217 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:42.993198 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bsdj7_18a538b6-74bf-4c7a-a451-3dda824187cb/init-textfile/0.log" Jun 15 03:54:43.469531 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:43.469501 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-864fcfc6f9-7pmwj_a737eeb8-bcbb-46db-952b-058c1a22710b/prometheus-operator/0.log" Jun 15 03:54:43.491932 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:43.491912 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-864fcfc6f9-7pmwj_a737eeb8-bcbb-46db-952b-058c1a22710b/kube-rbac-proxy/0.log" Jun 15 03:54:43.542912 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:43.542888 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5cf667749c-7hkl2_35718fb0-c60a-4206-a17f-746409b06664/prometheus-operator-admission-webhook/0.log" Jun 15 03:54:43.588493 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:43.588476 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-769944fdf-sf47j_58d07ff6-e115-4085-9eea-ad6bb1baf893/telemeter-client/0.log" Jun 15 03:54:43.623565 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:43.623541 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-769944fdf-sf47j_58d07ff6-e115-4085-9eea-ad6bb1baf893/reload/0.log" Jun 15 03:54:43.651885 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:43.651867 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-769944fdf-sf47j_58d07ff6-e115-4085-9eea-ad6bb1baf893/kube-rbac-proxy/0.log" Jun 15 03:54:43.695507 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:43.695485 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c674f96b-vw7b9_e4f343af-f13b-4775-80b0-766f80a17203/thanos-query/0.log" Jun 15 03:54:43.725238 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:43.725175 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c674f96b-vw7b9_e4f343af-f13b-4775-80b0-766f80a17203/kube-rbac-proxy-web/0.log" Jun 15 03:54:43.758056 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:43.758023 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c674f96b-vw7b9_e4f343af-f13b-4775-80b0-766f80a17203/kube-rbac-proxy/0.log" Jun 15 03:54:43.792051 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:43.792030 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c674f96b-vw7b9_e4f343af-f13b-4775-80b0-766f80a17203/prom-label-proxy/0.log" Jun 15 03:54:43.815599 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:43.815571 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c674f96b-vw7b9_e4f343af-f13b-4775-80b0-766f80a17203/kube-rbac-proxy-rules/0.log" Jun 15 03:54:43.837253 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:43.837235 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c674f96b-vw7b9_e4f343af-f13b-4775-80b0-766f80a17203/kube-rbac-proxy-metrics/0.log" Jun 15 03:54:45.157562 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:45.157520 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-867b76cb74-n9b2r_cf2910da-c1b3-4520-92be-4e1b96f6d878/networking-console-plugin/0.log" Jun 15 03:54:47.156459 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.156429 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-bd4ccd799-t6php_4ae9d1ef-8309-47e6-a27d-e57f590e0b2b/volume-data-source-validator/0.log" Jun 15 03:54:47.525935 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.525861 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2"] Jun 15 03:54:47.526297 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.526282 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cce0e4ea-d1b5-489b-9976-fc997f7a17b2" containerName="cleanup" Jun 15 03:54:47.526297 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.526297 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cce0e4ea-d1b5-489b-9976-fc997f7a17b2" containerName="cleanup" Jun 15 03:54:47.526490 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.526385 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cce0e4ea-d1b5-489b-9976-fc997f7a17b2" containerName="cleanup" Jun 15 03:54:47.529571 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.529554 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.532151 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.532130 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t4t9s\"/\"kube-root-ca.crt\"" Jun 15 03:54:47.533384 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.533364 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t4t9s\"/\"default-dockercfg-n4l8b\"" Jun 15 03:54:47.533485 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.533364 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t4t9s\"/\"openshift-service-ca.crt\"" Jun 15 03:54:47.541072 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.541051 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2"] Jun 15 03:54:47.669452 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.669422 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5e2693c-6e49-4c5e-945a-f1876a1063cc-lib-modules\") pod \"perf-node-gather-daemonset-mlns2\" (UID: \"d5e2693c-6e49-4c5e-945a-f1876a1063cc\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.669634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.669484 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5e2693c-6e49-4c5e-945a-f1876a1063cc-podres\") pod \"perf-node-gather-daemonset-mlns2\" (UID: \"d5e2693c-6e49-4c5e-945a-f1876a1063cc\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.669634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.669505 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzbqq\" (UniqueName: \"kubernetes.io/projected/d5e2693c-6e49-4c5e-945a-f1876a1063cc-kube-api-access-gzbqq\") pod \"perf-node-gather-daemonset-mlns2\" (UID: \"d5e2693c-6e49-4c5e-945a-f1876a1063cc\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.669634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.669526 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5e2693c-6e49-4c5e-945a-f1876a1063cc-sys\") pod \"perf-node-gather-daemonset-mlns2\" (UID: \"d5e2693c-6e49-4c5e-945a-f1876a1063cc\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.669634 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.669554 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5e2693c-6e49-4c5e-945a-f1876a1063cc-proc\") pod \"perf-node-gather-daemonset-mlns2\" (UID: \"d5e2693c-6e49-4c5e-945a-f1876a1063cc\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.770897 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.770866 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5e2693c-6e49-4c5e-945a-f1876a1063cc-podres\") pod \"perf-node-gather-daemonset-mlns2\" (UID: \"d5e2693c-6e49-4c5e-945a-f1876a1063cc\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.770897 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.770901 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzbqq\" (UniqueName: \"kubernetes.io/projected/d5e2693c-6e49-4c5e-945a-f1876a1063cc-kube-api-access-gzbqq\") pod \"perf-node-gather-daemonset-mlns2\" (UID: \"d5e2693c-6e49-4c5e-945a-f1876a1063cc\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.771129 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.770923 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5e2693c-6e49-4c5e-945a-f1876a1063cc-sys\") pod \"perf-node-gather-daemonset-mlns2\" (UID: \"d5e2693c-6e49-4c5e-945a-f1876a1063cc\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.771129 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.770955 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5e2693c-6e49-4c5e-945a-f1876a1063cc-proc\") pod \"perf-node-gather-daemonset-mlns2\" (UID: \"d5e2693c-6e49-4c5e-945a-f1876a1063cc\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.771129 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.770977 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5e2693c-6e49-4c5e-945a-f1876a1063cc-lib-modules\") pod \"perf-node-gather-daemonset-mlns2\" (UID: \"d5e2693c-6e49-4c5e-945a-f1876a1063cc\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.771129 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.771017 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5e2693c-6e49-4c5e-945a-f1876a1063cc-sys\") pod \"perf-node-gather-daemonset-mlns2\" (UID: \"d5e2693c-6e49-4c5e-945a-f1876a1063cc\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.771129 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.771055 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5e2693c-6e49-4c5e-945a-f1876a1063cc-podres\") pod \"perf-node-gather-daemonset-mlns2\" (UID: \"d5e2693c-6e49-4c5e-945a-f1876a1063cc\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.771129 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.771104 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5e2693c-6e49-4c5e-945a-f1876a1063cc-proc\") pod \"perf-node-gather-daemonset-mlns2\" (UID: \"d5e2693c-6e49-4c5e-945a-f1876a1063cc\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.771129 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.771129 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5e2693c-6e49-4c5e-945a-f1876a1063cc-lib-modules\") pod \"perf-node-gather-daemonset-mlns2\" (UID: \"d5e2693c-6e49-4c5e-945a-f1876a1063cc\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.778578 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.778517 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzbqq\" (UniqueName: \"kubernetes.io/projected/d5e2693c-6e49-4c5e-945a-f1876a1063cc-kube-api-access-gzbqq\") pod \"perf-node-gather-daemonset-mlns2\" (UID: \"d5e2693c-6e49-4c5e-945a-f1876a1063cc\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.840328 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.840294 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:47.975353 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:47.975326 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2"] Jun 15 03:54:47.976409 ip-10-0-139-167 kubenswrapper[2560]: W0615 03:54:47.976382 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd5e2693c_6e49_4c5e_945a_f1876a1063cc.slice/crio-f6c4ac9d115976e9ec10b72b5f5b6723bd16b35e7bc1f9092543af77b419bcae WatchSource:0}: Error finding container f6c4ac9d115976e9ec10b72b5f5b6723bd16b35e7bc1f9092543af77b419bcae: Status 404 returned error can't find the container with id f6c4ac9d115976e9ec10b72b5f5b6723bd16b35e7bc1f9092543af77b419bcae Jun 15 03:54:48.356579 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:48.356495 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5zmqd_f10bedca-8f8a-4db1-9b58-8d3d854ece5d/dns/0.log" Jun 15 03:54:48.414781 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:48.414754 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5zmqd_f10bedca-8f8a-4db1-9b58-8d3d854ece5d/kube-rbac-proxy/0.log" Jun 15 03:54:48.613344 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:48.613276 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zq7r8_48bd8c21-f7a6-47fb-adda-1f4d03b37c64/dns-node-resolver/0.log" Jun 15 03:54:48.629402 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:48.629375 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" event={"ID":"d5e2693c-6e49-4c5e-945a-f1876a1063cc","Type":"ContainerStarted","Data":"bd5700cb3cb2cbf107115913fd6868fada11994fd2a66a70cc085f032e7deab5"} Jun 15 03:54:48.629521 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:48.629423 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" event={"ID":"d5e2693c-6e49-4c5e-945a-f1876a1063cc","Type":"ContainerStarted","Data":"f6c4ac9d115976e9ec10b72b5f5b6723bd16b35e7bc1f9092543af77b419bcae"} Jun 15 03:54:48.629579 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:48.629524 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:54:48.646917 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:48.646875 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" podStartSLOduration=1.6468619009999999 podStartE2EDuration="1.646861901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:54:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:54:48.643959391 +0000 UTC m=+2766.800573590" watchObservedRunningTime="2026-06-15 03:54:48.646861901 +0000 UTC m=+2766.803476109" Jun 15 03:54:49.193051 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:49.193022 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vzjh4_5c55e28b-e913-482a-b2ef-e9bf7d2e78e8/node-ca/0.log" Jun 15 03:54:50.279359 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:50.279331 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-l7p2l_0429504b-413e-45fb-ae04-8345e31b1f31/discovery/0.log" Jun 15 03:54:50.304535 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:50.304514 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7579ff4b78-jgtz4_77f6c16f-a5eb-4963-abb0-db8b2ad4db30/kube-auth-proxy/0.log" Jun 15 03:54:50.514366 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:50.514334 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55f78b5b6d-xfjzj_1429058a-5113-4842-83d5-1f57f1fee24a/router/0.log" Jun 15 03:54:51.177071 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:51.177039 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s7jcv_5dab88de-1c68-4f1e-a4a3-e6a92f7b46c5/serve-healthcheck-canary/0.log" Jun 15 03:54:51.762346 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:51.762313 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qmv4z_2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307/kube-rbac-proxy/0.log" Jun 15 03:54:51.783252 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:51.783210 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qmv4z_2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307/exporter/0.log" Jun 15 03:54:51.805038 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:51.805017 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qmv4z_2ce2fb1f-f3b9-4fca-a5ff-d7e43b11d307/extractor/0.log" Jun 15 03:54:54.020915 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:54.020886 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-694cb6497d-qpx9v_7749efc3-a41d-4aac-a75d-f28b4f1b9ec0/nginx/0.log" Jun 15 03:54:54.045628 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:54.045604 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-dbmcd_9bbcc475-5dd9-4830-bcd9-d3b0bebe58dc/manager/0.log" Jun 15 03:54:54.113501 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:54.113478 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-xmklr_3f22b40b-22cd-43b1-892b-f0fc87707156/manager/0.log" Jun 15 03:54:54.166854 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:54.166831 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29691570-4wm89_cce0e4ea-d1b5-489b-9976-fc997f7a17b2/cleanup/0.log" Jun 15 03:54:54.278783 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:54.278688 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-xsk7z_5570bf43-0431-4cd1-8c78-c02ff1ac57e2/server/0.log" Jun 15 03:54:54.330953 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:54.330929 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-jnphp_419a1975-d79b-4623-853b-bba16bc0e091/manager/0.log" Jun 15 03:54:54.513078 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:54.513054 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5799f64c8f-zd8s7_c12ee043-c8d5-4ad7-b955-bb125df4d5a0/manager/0.log" Jun 15 03:54:54.540103 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:54.540062 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5799f64c8f-zd8s7_c12ee043-c8d5-4ad7-b955-bb125df4d5a0/copy-manifests/0.log" Jun 15 03:54:54.564460 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:54.564433 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-kgpvn_f6f9f083-598d-424a-88d3-875c3d324f16/postgres/0.log" Jun 15 03:54:54.644953 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:54:54.644931 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-mlns2" Jun 15 03:55:00.095040 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:00.095003 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-l7p2l_0429504b-413e-45fb-ae04-8345e31b1f31/discovery/0.log" Jun 15 03:55:00.110646 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:00.110625 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7579ff4b78-jgtz4_77f6c16f-a5eb-4963-abb0-db8b2ad4db30/kube-auth-proxy/0.log" Jun 15 03:55:00.268011 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:00.267986 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55f78b5b6d-xfjzj_1429058a-5113-4842-83d5-1f57f1fee24a/router/0.log" Jun 15 03:55:01.482499 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:01.482467 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-l7p2l_0429504b-413e-45fb-ae04-8345e31b1f31/discovery/0.log" Jun 15 03:55:01.495609 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:01.495583 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7579ff4b78-jgtz4_77f6c16f-a5eb-4963-abb0-db8b2ad4db30/kube-auth-proxy/0.log" Jun 15 03:55:01.659796 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:01.659765 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55f78b5b6d-xfjzj_1429058a-5113-4842-83d5-1f57f1fee24a/router/0.log" Jun 15 03:55:03.344708 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:03.344678 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzn4l_bd7848c5-9bec-441f-bda7-a249bd472e1b/kube-multus-additional-cni-plugins/0.log" Jun 15 03:55:03.365830 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:03.365803 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzn4l_bd7848c5-9bec-441f-bda7-a249bd472e1b/egress-router-binary-copy/0.log" Jun 15 03:55:03.387016 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:03.386986 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzn4l_bd7848c5-9bec-441f-bda7-a249bd472e1b/cni-plugins/0.log" Jun 15 03:55:03.408607 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:03.408569 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-l7p2l_0429504b-413e-45fb-ae04-8345e31b1f31/discovery/0.log" Jun 15 03:55:03.408827 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:03.408810 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzn4l_bd7848c5-9bec-441f-bda7-a249bd472e1b/bond-cni-plugin/0.log" Jun 15 03:55:03.417425 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:03.417394 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7579ff4b78-jgtz4_77f6c16f-a5eb-4963-abb0-db8b2ad4db30/kube-auth-proxy/0.log" Jun 15 03:55:03.429623 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:03.429598 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzn4l_bd7848c5-9bec-441f-bda7-a249bd472e1b/routeoverride-cni/0.log" Jun 15 03:55:03.452602 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:03.452552 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzn4l_bd7848c5-9bec-441f-bda7-a249bd472e1b/whereabouts-cni-bincopy/0.log" Jun 15 03:55:03.474698 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:03.474664 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzn4l_bd7848c5-9bec-441f-bda7-a249bd472e1b/whereabouts-cni/0.log" Jun 15 03:55:03.560775 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:03.560743 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55f78b5b6d-xfjzj_1429058a-5113-4842-83d5-1f57f1fee24a/router/0.log" Jun 15 03:55:03.664012 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:03.663931 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-bz9qb_df0dc0ff-b7d7-4d79-ad9d-ee4d600e157f/kube-multus/0.log" Jun 15 03:55:03.743889 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:03.743863 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9z4f4_ff14c0e9-ddf7-4720-ac87-54a88354948e/network-metrics-daemon/0.log" Jun 15 03:55:03.764263 ip-10-0-139-167 kubenswrapper[2560]: I0615 03:55:03.764234 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9z4f4_ff14c0e9-ddf7-4720-ac87-54a88354948e/kube-rbac-proxy/0.log"