Jun 11 18:35:58.285471 ip-10-0-138-156 systemd[1]: Starting Kubernetes Kubelet... Jun 11 18:35:58.748718 ip-10-0-138-156 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 18:35:58.748718 ip-10-0-138-156 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 11 18:35:58.748718 ip-10-0-138-156 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 18:35:58.748718 ip-10-0-138-156 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 11 18:35:58.748718 ip-10-0-138-156 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 18:35:58.749884 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.749799 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 11 18:35:58.755451 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755435 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 18:35:58.755451 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755451 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755455 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755459 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755462 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755465 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755468 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755471 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755474 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755478 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755482 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755485 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755488 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755491 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755494 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755498 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755500 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755503 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755506 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755509 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755511 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 18:35:58.755523 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755514 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755516 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755519 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755522 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755524 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755527 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755530 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755533 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755535 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755538 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755543 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755547 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755550 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755552 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755555 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755558 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755561 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755565 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755567 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 18:35:58.756001 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755570 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755572 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755575 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755578 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755580 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755583 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755585 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755588 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755598 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755601 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755604 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755607 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755609 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755612 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755615 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755618 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755621 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755624 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755626 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 18:35:58.756498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755629 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755631 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755634 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755637 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755640 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755642 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755645 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755648 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755650 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755653 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755655 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755658 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755661 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755663 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755666 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755669 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755671 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755674 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755676 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755678 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 18:35:58.756980 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755681 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755683 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755686 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755690 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755693 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755697 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755699 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755702 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755706 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755710 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755712 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755715 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755718 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755720 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755723 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755725 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755728 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755730 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755733 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 18:35:58.757515 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.755735 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756441 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756447 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756450 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756454 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756458 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756463 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756466 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756469 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756472 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756475 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756477 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756480 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756483 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756485 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756488 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756490 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756493 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756495 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 18:35:58.758005 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756499 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756503 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756506 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756509 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756512 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756515 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756517 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756520 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756522 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756525 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756527 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756530 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756533 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756535 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756538 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756540 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756543 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756545 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756548 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756551 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 18:35:58.758498 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756554 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756557 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756559 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756562 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756564 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756567 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756570 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756572 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756574 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756577 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756580 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756583 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756585 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756588 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756591 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756593 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756596 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756598 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756601 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756603 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 18:35:58.759025 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756606 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756610 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756613 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756616 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756619 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756622 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756624 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756627 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756630 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756632 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756637 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756640 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756642 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756645 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756648 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756650 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756653 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756655 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756658 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 18:35:58.759524 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756661 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756663 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756666 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756669 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756671 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756674 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756677 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756681 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756685 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756688 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756690 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756692 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756695 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756698 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756701 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756703 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756706 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756709 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756712 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756714 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 18:35:58.759979 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756717 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.756719 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757081 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757091 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757094 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757097 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757101 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757103 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757107 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757109 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757112 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757114 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757117 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757119 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757122 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757125 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757127 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757130 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757133 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757135 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 18:35:58.760480 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757138 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757140 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757143 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757145 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757148 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757150 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757153 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757156 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757158 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757160 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757164 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757167 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757184 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757187 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757190 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757194 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757196 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757199 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757201 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757205 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 18:35:58.760955 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757207 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757210 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757213 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757216 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757219 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757222 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757224 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757227 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757230 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757233 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757236 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757238 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757241 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757244 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757246 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757249 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757251 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757254 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757257 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757259 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 18:35:58.761555 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757262 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757264 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757267 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757270 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757272 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757275 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757277 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757281 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757284 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757287 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757289 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757292 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757296 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757298 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757301 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757303 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757305 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757308 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757311 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757313 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 18:35:58.762036 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757318 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757321 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757325 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757328 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757332 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757335 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757337 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757340 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757343 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757346 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757349 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757351 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757354 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757357 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757360 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757362 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757365 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757368 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757372 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 18:35:58.762563 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757377 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.757380 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757454 2561 flags.go:64] FLAG: --address="0.0.0.0" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757460 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757466 2561 flags.go:64] FLAG: --anonymous-auth="true" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757470 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757475 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757478 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757482 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757487 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757490 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757493 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757496 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757501 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757504 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757507 2561 flags.go:64] FLAG: --cgroup-root="" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757510 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757513 2561 flags.go:64] FLAG: --client-ca-file="" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757515 2561 flags.go:64] FLAG: --cloud-provider="external" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757518 2561 flags.go:64] FLAG: --cluster-dns="[]" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757523 2561 flags.go:64] FLAG: --cluster-domain="" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757526 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757529 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757532 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 11 18:35:58.763037 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757536 2561 flags.go:64] FLAG: --container-log-max-files="5" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757540 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757543 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757546 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757549 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757552 2561 flags.go:64] FLAG: --contention-profiling="false" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757555 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757557 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757561 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757564 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757568 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757571 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757574 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757577 2561 flags.go:64] FLAG: --enable-load-reader="false" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757580 2561 flags.go:64] FLAG: --enable-server="true" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757583 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757588 2561 flags.go:64] FLAG: --event-burst="100" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757591 2561 flags.go:64] FLAG: --event-qps="50" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757593 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757597 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757600 2561 flags.go:64] FLAG: --eviction-hard="" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757604 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757607 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757610 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757613 2561 flags.go:64] FLAG: --eviction-soft="" Jun 11 18:35:58.763627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757616 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757619 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757622 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757625 2561 flags.go:64] FLAG: --experimental-mounter-path="" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757627 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757630 2561 flags.go:64] FLAG: --fail-swap-on="true" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757633 2561 flags.go:64] FLAG: --feature-gates="" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757637 2561 flags.go:64] FLAG: --file-check-frequency="20s" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757640 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757642 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757646 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757649 2561 flags.go:64] FLAG: --healthz-port="10248" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757652 2561 flags.go:64] FLAG: --help="false" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757655 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-138-156.ec2.internal" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757658 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757661 2561 flags.go:64] FLAG: --http-check-frequency="20s" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757664 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757667 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757670 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757673 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757676 2561 flags.go:64] FLAG: --image-service-endpoint="" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757678 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757681 2561 flags.go:64] FLAG: --kube-api-burst="100" Jun 11 18:35:58.764238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757685 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757688 2561 flags.go:64] FLAG: --kube-api-qps="50" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757690 2561 flags.go:64] FLAG: --kube-reserved="" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757693 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757696 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757700 2561 flags.go:64] FLAG: --kubelet-cgroups="" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757703 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757705 2561 flags.go:64] FLAG: --lock-file="" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757708 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757711 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757714 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757719 2561 flags.go:64] FLAG: --log-json-split-stream="false" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757722 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757724 2561 flags.go:64] FLAG: --log-text-split-stream="false" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757727 2561 flags.go:64] FLAG: --logging-format="text" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757730 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757733 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757736 2561 flags.go:64] FLAG: --manifest-url="" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757739 2561 flags.go:64] FLAG: --manifest-url-header="" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757743 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757747 2561 flags.go:64] FLAG: --max-open-files="1000000" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757751 2561 flags.go:64] FLAG: --max-pods="110" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757754 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757756 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757759 2561 flags.go:64] FLAG: --memory-manager-policy="None" Jun 11 18:35:58.764845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757763 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757766 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757768 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757771 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757779 2561 flags.go:64] FLAG: --node-status-max-images="50" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757781 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757784 2561 flags.go:64] FLAG: --oom-score-adj="-999" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757788 2561 flags.go:64] FLAG: --pod-cidr="" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757791 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4a2e90c8ef7d783e4d635e24fb09435f5e1f6e954583512c0944f9e57bc77b52" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757797 2561 flags.go:64] FLAG: --pod-manifest-path="" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757800 2561 flags.go:64] FLAG: --pod-max-pids="-1" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757803 2561 flags.go:64] FLAG: --pods-per-core="0" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757806 2561 flags.go:64] FLAG: --port="10250" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757810 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757813 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0289cd0185dbfbf69" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757816 2561 flags.go:64] FLAG: --qos-reserved="" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757819 2561 flags.go:64] FLAG: --read-only-port="10255" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757822 2561 flags.go:64] FLAG: --register-node="true" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757825 2561 flags.go:64] FLAG: --register-with-taints="" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757828 2561 flags.go:64] FLAG: --registry-burst="10" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757831 2561 flags.go:64] FLAG: --registry-qps="5" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757834 2561 flags.go:64] FLAG: --reserved-cpus="" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757837 2561 flags.go:64] FLAG: --reserved-memory="" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757840 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757843 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 11 18:35:58.765888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757846 2561 flags.go:64] FLAG: --rotate-certificates="false" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757848 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757851 2561 flags.go:64] FLAG: --runonce="false" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757854 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757857 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757860 2561 flags.go:64] FLAG: --seccomp-default="false" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757863 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757866 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757869 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757872 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757875 2561 flags.go:64] FLAG: --storage-driver-password="root" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757878 2561 flags.go:64] FLAG: --storage-driver-secure="false" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757880 2561 flags.go:64] FLAG: --storage-driver-table="stats" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757883 2561 flags.go:64] FLAG: --storage-driver-user="root" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757886 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757889 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757892 2561 flags.go:64] FLAG: --system-cgroups="" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757895 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757900 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757903 2561 flags.go:64] FLAG: --tls-cert-file="" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757907 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757911 2561 flags.go:64] FLAG: --tls-min-version="" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757914 2561 flags.go:64] FLAG: --tls-private-key-file="" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757917 2561 flags.go:64] FLAG: --topology-manager-policy="none" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757919 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 11 18:35:58.767046 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757922 2561 flags.go:64] FLAG: --topology-manager-scope="container" Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757925 2561 flags.go:64] FLAG: --v="2" Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757929 2561 flags.go:64] FLAG: --version="false" Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757933 2561 flags.go:64] FLAG: --vmodule="" Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757936 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.757939 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758057 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758061 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758064 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758067 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758069 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758072 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758075 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758079 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758082 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758084 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758087 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758090 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758092 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758095 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758098 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 18:35:58.768191 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758100 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758103 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758106 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758109 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758111 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758114 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758117 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758120 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758122 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758125 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758127 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758130 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758134 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758137 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758140 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758143 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758145 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758148 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758150 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 18:35:58.769074 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758153 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758155 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758158 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758160 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758163 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758167 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758186 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758189 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758191 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758194 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758196 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758199 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758202 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758204 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758207 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758209 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758212 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758215 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758218 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758221 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 18:35:58.769630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758223 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758226 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758229 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758231 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758234 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758237 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758239 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758242 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758245 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758247 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758250 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758252 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758255 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758258 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758260 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758263 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758265 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758269 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758272 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758274 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 18:35:58.770387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758278 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758282 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758285 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758288 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758290 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758294 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758297 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758299 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758302 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758305 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758309 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758312 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758314 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758317 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758319 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758322 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758324 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758327 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758329 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758333 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 18:35:58.771233 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758336 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 18:35:58.771849 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758339 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 18:35:58.771849 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758342 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 18:35:58.771849 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758344 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 18:35:58.771849 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.758346 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 18:35:58.771849 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.759280 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 18:35:58.771849 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.767494 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 11 18:35:58.771849 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.767514 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 11 18:35:58.771849 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.767544 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 11 18:35:58.771849 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.767552 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 11 18:35:58.771849 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767646 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 18:35:58.771849 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767654 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 18:35:58.771849 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767660 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 18:35:58.771849 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767665 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 18:35:58.771849 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767670 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 18:35:58.771849 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767675 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767680 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767685 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767690 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767694 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767699 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767703 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767721 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767726 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767731 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767735 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767739 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767744 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767748 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767753 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767757 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767762 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767768 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767775 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767779 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 18:35:58.772272 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767783 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767787 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767794 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767800 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767805 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767810 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767814 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767819 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767823 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767828 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767833 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767838 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767843 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767848 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767852 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767856 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767860 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767864 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767869 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 18:35:58.772866 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767873 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767885 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767890 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767895 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767899 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767904 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767908 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767913 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767917 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767921 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767925 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767930 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767934 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767938 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767942 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767947 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767951 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767956 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767960 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767964 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 18:35:58.773376 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767968 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767973 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767977 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767981 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767985 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767989 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767994 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.767998 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768002 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768006 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768010 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768015 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768019 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768024 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768042 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768047 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768054 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768060 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768064 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 18:35:58.773858 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768069 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 18:35:58.774471 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768074 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 18:35:58.774471 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768078 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 18:35:58.774471 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768082 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 18:35:58.774471 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768087 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 18:35:58.774471 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768091 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 18:35:58.774471 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768095 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 18:35:58.774471 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768099 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 18:35:58.774471 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768104 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 18:35:58.774471 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768109 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 18:35:58.774471 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768115 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 18:35:58.774471 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768119 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 18:35:58.774471 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768124 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 18:35:58.774471 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768128 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 18:35:58.774471 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768132 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 18:35:58.774471 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768136 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 18:35:58.774867 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.768144 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 18:35:58.774867 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768384 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 18:35:58.774867 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768395 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 18:35:58.774867 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768400 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 18:35:58.774867 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768406 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 18:35:58.774867 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768410 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 18:35:58.774867 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768414 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 18:35:58.774867 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768418 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 18:35:58.774867 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768423 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 18:35:58.774867 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768427 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 18:35:58.774867 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768431 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 18:35:58.774867 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768435 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 18:35:58.774867 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768448 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 18:35:58.774867 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768452 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 18:35:58.774867 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768457 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768461 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768465 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768470 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768474 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768478 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768483 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768487 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768492 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768496 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768500 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768505 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768509 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768513 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768517 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768522 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768526 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768530 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768534 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768539 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 18:35:58.775266 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768543 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768547 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768552 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768557 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768561 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768565 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768572 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768578 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768583 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768587 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768591 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768603 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768608 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768612 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768616 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768621 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768625 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768629 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768634 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 18:35:58.775752 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768638 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768642 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768646 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768650 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768654 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768658 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768663 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768667 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768672 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768677 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768681 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768685 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768691 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768696 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768701 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768706 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768711 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768715 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768719 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768724 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 18:35:58.776232 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768728 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768733 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768737 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768741 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768746 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768758 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768763 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768767 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768771 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768776 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768780 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768784 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768788 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768792 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768796 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768800 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768805 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768809 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768813 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768818 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 18:35:58.776710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768822 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 18:35:58.777210 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768826 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 18:35:58.777210 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768830 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 18:35:58.777210 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768834 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 18:35:58.777210 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768840 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 18:35:58.777210 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768845 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 18:35:58.777210 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:58.768849 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 18:35:58.777210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.768858 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 18:35:58.777210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.769031 2561 server.go:962] "Client rotation is on, will bootstrap in background" Jun 11 18:35:58.777210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.772338 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 11 18:35:58.777210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.774163 2561 server.go:1019] "Starting client certificate rotation" Jun 11 18:35:58.777210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.774283 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 18:35:58.777210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.774333 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 18:35:58.777517 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.777258 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 18:35:58.782408 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.782384 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 18:35:58.793214 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.793198 2561 log.go:25] "Validated CRI v1 runtime API" Jun 11 18:35:58.796856 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.796843 2561 log.go:25] "Validated CRI v1 image API" Jun 11 18:35:58.798405 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.798386 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 11 18:35:58.801127 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.801109 2561 fs.go:135] Filesystem UUIDs: map[0c548136-fcb5-410b-9a58-ba717e680cb2:/dev/nvme0n1p3 79bc623b-4425-4b25-8863-61bfc922d442:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jun 11 18:35:58.801203 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.801126 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 11 18:35:58.806464 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.806355 2561 manager.go:216] Machine: {Timestamp:2026-06-11 18:35:58.804919439 +0000 UTC m=+0.372992911 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097792 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24ef0a7960261f3e25e8263c84a059 SystemUUID:ec24ef0a-7960-261f-3e25-e8263c84a059 BootID:d3e63a9d-f9f9-4760-beb4-368ad43e73bd Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:29:4b:7a:d1:67 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:29:4b:7a:d1:67 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:a3:6f:dd:7a:2d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 11 18:35:58.806464 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.806463 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 11 18:35:58.806576 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.806539 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.120.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260608-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 11 18:35:58.810158 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.810130 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 11 18:35:58.810315 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.810163 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-156.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 11 18:35:58.811704 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.811692 2561 topology_manager.go:138] "Creating topology manager with none policy" Jun 11 18:35:58.811743 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.811709 2561 container_manager_linux.go:309] "Creating device plugin manager" Jun 11 18:35:58.811743 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.811717 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 18:35:58.812524 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.812510 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 18:35:58.812583 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.812568 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 18:35:58.814421 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.814409 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 11 18:35:58.819136 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.819124 2561 state_mem.go:36] "Initialized new in-memory state store" Jun 11 18:35:58.819256 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.819247 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 11 18:35:58.819286 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.819277 2561 kubelet.go:486] "Attempting to sync node with API server" Jun 11 18:35:58.819313 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.819287 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 11 18:35:58.819313 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.819297 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 11 18:35:58.819313 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.819307 2561 kubelet.go:398] "Adding apiserver pod source" Jun 11 18:35:58.819405 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.819316 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 11 18:35:58.819665 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.819652 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 18:35:58.819701 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.819674 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 18:35:58.822107 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.822081 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 11 18:35:58.823887 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.823874 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 11 18:35:58.823941 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.823905 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 11 18:35:58.824059 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.824049 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 11 18:35:58.824088 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.824064 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 11 18:35:58.824088 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.824071 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 11 18:35:58.824088 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.824077 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 11 18:35:58.824088 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.824084 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 11 18:35:58.824088 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.824089 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 11 18:35:58.824721 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.824712 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 11 18:35:58.824754 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.824729 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 11 18:35:58.824754 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.824736 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 11 18:35:58.824754 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.824741 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 11 18:35:58.824754 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.824753 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 11 18:35:58.824851 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.824760 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 11 18:35:58.825659 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.825650 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 11 18:35:58.825696 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.825659 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 11 18:35:58.828806 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.828792 2561 server.go:1268] "Started kubelet" Jun 11 18:35:58.828914 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.828883 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 11 18:35:58.832186 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.832114 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 11 18:35:58.832264 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.832214 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 11 18:35:58.833571 ip-10-0-138-156 systemd[1]: Started Kubernetes Kubelet. Jun 11 18:35:58.834316 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.834300 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 11 18:35:58.835976 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.835958 2561 server.go:310] "Adding debug handlers to kubelet server" Jun 11 18:35:58.839040 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.839023 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-156.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 11 18:35:58.839781 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.839760 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 11 18:35:58.839781 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.839762 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-156.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 11 18:35:58.840414 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.839298 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-156.ec2.internal.18b81aa761dceec4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-156.ec2.internal,UID:ip-10-0-138-156.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-156.ec2.internal,},FirstTimestamp:2026-06-11 18:35:58.828760772 +0000 UTC m=+0.396834243,LastTimestamp:2026-06-11 18:35:58.828760772 +0000 UTC m=+0.396834243,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-156.ec2.internal,}" Jun 11 18:35:58.840764 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.840738 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 11 18:35:58.842279 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.842264 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 11 18:35:58.842339 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.842266 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 11 18:35:58.842413 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.842393 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-156.ec2.internal\" not found" Jun 11 18:35:58.843075 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.843056 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 11 18:35:58.843075 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.843078 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 11 18:35:58.843232 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.843056 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 11 18:35:58.843232 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.843201 2561 reconstruct.go:97] "Volume reconstruction finished" Jun 11 18:35:58.843232 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.843209 2561 reconciler.go:29] "Reconciler: start to sync state" Jun 11 18:35:58.843358 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.843343 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 11 18:35:58.843408 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.843359 2561 factory.go:55] Registering systemd factory Jun 11 18:35:58.843408 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.843365 2561 factory.go:223] Registration of the systemd container factory successfully Jun 11 18:35:58.843563 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.843552 2561 factory.go:153] Registering CRI-O factory Jun 11 18:35:58.843563 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.843563 2561 factory.go:223] Registration of the crio container factory successfully Jun 11 18:35:58.843630 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.843582 2561 factory.go:103] Registering Raw factory Jun 11 18:35:58.843630 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.843598 2561 manager.go:1196] Started watching for new ooms in manager Jun 11 18:35:58.843999 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.843988 2561 manager.go:349] Starting recovery of all containers Jun 11 18:35:58.855425 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.855272 2561 manager.go:354] Recovery completed Jun 11 18:35:58.856740 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.856710 2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-156.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 11 18:35:58.856830 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.856720 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 11 18:35:58.857207 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.857151 2561 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jun 11 18:35:58.860343 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.860277 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 18:35:58.866563 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.866541 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientMemory" Jun 11 18:35:58.866646 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.866590 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 18:35:58.866646 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.866600 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientPID" Jun 11 18:35:58.867216 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.867203 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 11 18:35:58.867216 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.867215 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 11 18:35:58.867317 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.867222 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 18:35:58.867317 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.867230 2561 state_mem.go:36] "Initialized new in-memory state store" Jun 11 18:35:58.868423 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.868410 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientMemory" Jun 11 18:35:58.868491 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.868434 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 18:35:58.868491 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.868444 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientPID" Jun 11 18:35:58.869459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.869446 2561 policy_none.go:49] "None policy: Start" Jun 11 18:35:58.869508 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.869463 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 11 18:35:58.869508 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.869474 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 11 18:35:58.871446 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.871435 2561 policy_none.go:47] "Start" Jun 11 18:35:58.878075 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.877997 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-156.ec2.internal.18b81aa7641dbf4d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-156.ec2.internal,UID:ip-10-0-138-156.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-156.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-156.ec2.internal,},FirstTimestamp:2026-06-11 18:35:58.866562893 +0000 UTC m=+0.434636367,LastTimestamp:2026-06-11 18:35:58.866562893 +0000 UTC m=+0.434636367,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-156.ec2.internal,}" Jun 11 18:35:58.890119 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.890054 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-156.ec2.internal.18b81aa7641e3db3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-156.ec2.internal,UID:ip-10-0-138-156.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-156.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-156.ec2.internal,},FirstTimestamp:2026-06-11 18:35:58.866595251 +0000 UTC m=+0.434668722,LastTimestamp:2026-06-11 18:35:58.866595251 +0000 UTC m=+0.434668722,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-156.ec2.internal,}" Jun 11 18:35:58.895058 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.895038 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-phfv5" Jun 11 18:35:58.900316 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.900245 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-156.ec2.internal.18b81aa7641e7180 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-156.ec2.internal,UID:ip-10-0-138-156.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-156.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-156.ec2.internal,},FirstTimestamp:2026-06-11 18:35:58.866608512 +0000 UTC m=+0.434681982,LastTimestamp:2026-06-11 18:35:58.866608512 +0000 UTC m=+0.434681982,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-156.ec2.internal,}" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.904399 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-phfv5" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.908190 2561 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-156.ec2.internal.18b81aa7641dbf4d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-156.ec2.internal.18b81aa7641dbf4d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-156.ec2.internal,UID:ip-10-0-138-156.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-156.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-156.ec2.internal,},FirstTimestamp:2026-06-11 18:35:58.866562893 +0000 UTC m=+0.434636367,LastTimestamp:2026-06-11 18:35:58.868422059 +0000 UTC m=+0.436495529,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-156.ec2.internal,}" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.910778 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.911867 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.911902 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.911923 2561 kubelet.go:2448] "Starting kubelet main sync loop" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.911955 2561 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.915834 2561 manager.go:337] "Starting Device Plugin manager" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.915865 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.915878 2561 server.go:87] "Starting device plugin registration server" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.916134 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.916145 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.916226 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.916308 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.916315 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.916800 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:58.916838 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-156.ec2.internal\" not found" Jun 11 18:35:58.922210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:58.921546 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 18:35:59.012884 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.012781 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-156.ec2.internal"] Jun 11 18:35:59.012884 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.012878 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 18:35:59.014608 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.014593 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientMemory" Jun 11 18:35:59.014694 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.014621 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 18:35:59.014694 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.014643 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientPID" Jun 11 18:35:59.016727 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.016714 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 18:35:59.016989 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.016975 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 18:35:59.017072 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.017060 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.017113 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.017084 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 18:35:59.017492 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.017477 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientMemory" Jun 11 18:35:59.017581 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.017501 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 18:35:59.017581 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.017477 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientMemory" Jun 11 18:35:59.017581 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.017535 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 18:35:59.017581 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.017515 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientPID" Jun 11 18:35:59.017581 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.017552 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientPID" Jun 11 18:35:59.017814 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.017607 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.018960 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.018939 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientMemory" Jun 11 18:35:59.019047 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.018969 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 18:35:59.019047 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.018996 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientPID" Jun 11 18:35:59.019861 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.019845 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.019939 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.019872 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 18:35:59.020955 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.020938 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientMemory" Jun 11 18:35:59.021028 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.020960 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 18:35:59.021028 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.020971 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientPID" Jun 11 18:35:59.043792 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.043762 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f345c025a1c78c555e7e71fabd161286-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal\" (UID: \"f345c025a1c78c555e7e71fabd161286\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.043929 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.043798 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f345c025a1c78c555e7e71fabd161286-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal\" (UID: \"f345c025a1c78c555e7e71fabd161286\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.045136 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:59.045115 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-156.ec2.internal\" not found" node="ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.049441 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:59.049425 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-156.ec2.internal\" not found" node="ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.063506 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.063481 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.063506 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:59.063507 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-156.ec2.internal\": node \"ip-10-0-138-156.ec2.internal\" not found" Jun 11 18:35:59.102474 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:59.102445 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-156.ec2.internal\" not found" Jun 11 18:35:59.144272 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.144238 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f345c025a1c78c555e7e71fabd161286-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal\" (UID: \"f345c025a1c78c555e7e71fabd161286\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.144272 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.144268 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f345c025a1c78c555e7e71fabd161286-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal\" (UID: \"f345c025a1c78c555e7e71fabd161286\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.144272 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.144275 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f345c025a1c78c555e7e71fabd161286-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal\" (UID: \"f345c025a1c78c555e7e71fabd161286\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.144508 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.144301 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/536524fd725acf153e94591109a49596-config\") pod \"kube-apiserver-proxy-ip-10-0-138-156.ec2.internal\" (UID: \"536524fd725acf153e94591109a49596\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.144508 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.144326 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f345c025a1c78c555e7e71fabd161286-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal\" (UID: \"f345c025a1c78c555e7e71fabd161286\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.203291 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:59.203258 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-156.ec2.internal\" not found" Jun 11 18:35:59.244449 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.244420 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/536524fd725acf153e94591109a49596-config\") pod \"kube-apiserver-proxy-ip-10-0-138-156.ec2.internal\" (UID: \"536524fd725acf153e94591109a49596\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.244512 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.244475 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/536524fd725acf153e94591109a49596-config\") pod \"kube-apiserver-proxy-ip-10-0-138-156.ec2.internal\" (UID: \"536524fd725acf153e94591109a49596\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.303583 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:59.303508 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-156.ec2.internal\" not found" Jun 11 18:35:59.348198 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.348163 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.354932 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.354913 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-156.ec2.internal" Jun 11 18:35:59.404332 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:59.404297 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-156.ec2.internal\" not found" Jun 11 18:35:59.504812 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:59.504777 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-156.ec2.internal\" not found" Jun 11 18:35:59.605405 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:59.605339 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-156.ec2.internal\" not found" Jun 11 18:35:59.705875 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:59.705843 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-156.ec2.internal\" not found" Jun 11 18:35:59.715444 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.715412 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 18:35:59.773063 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.773032 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 11 18:35:59.773645 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.773197 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 18:35:59.773645 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.773214 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 18:35:59.806320 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:59.806290 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-156.ec2.internal\" not found" Jun 11 18:35:59.863126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.863067 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 18:35:59.903630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:59.903587 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod536524fd725acf153e94591109a49596.slice/crio-0131dea0cd7b6a2515d7c91647ab1db898a7a67d6d3e7175d62466c69067f38d WatchSource:0}: Error finding container 0131dea0cd7b6a2515d7c91647ab1db898a7a67d6d3e7175d62466c69067f38d: Status 404 returned error can't find the container with id 0131dea0cd7b6a2515d7c91647ab1db898a7a67d6d3e7175d62466c69067f38d Jun 11 18:35:59.906410 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:35:59.906392 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-156.ec2.internal\" not found" Jun 11 18:35:59.908283 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.908261 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 18:35:59.910601 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.910573 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-10 18:30:58 +0000 UTC" deadline="2028-02-17 13:43:38.366682688 +0000 UTC" Jun 11 18:35:59.910710 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.910602 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14779h7m38.456084995s" Jun 11 18:35:59.911045 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:35:59.911021 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf345c025a1c78c555e7e71fabd161286.slice/crio-259e0ff84165175f27eaffcebd18c719870adcd6f6302abf7f2c2a4470a76ed6 WatchSource:0}: Error finding container 259e0ff84165175f27eaffcebd18c719870adcd6f6302abf7f2c2a4470a76ed6: Status 404 returned error can't find the container with id 259e0ff84165175f27eaffcebd18c719870adcd6f6302abf7f2c2a4470a76ed6 Jun 11 18:35:59.915043 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.915007 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal" event={"ID":"f345c025a1c78c555e7e71fabd161286","Type":"ContainerStarted","Data":"259e0ff84165175f27eaffcebd18c719870adcd6f6302abf7f2c2a4470a76ed6"} Jun 11 18:35:59.915882 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.915864 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-156.ec2.internal" event={"ID":"536524fd725acf153e94591109a49596","Type":"ContainerStarted","Data":"0131dea0cd7b6a2515d7c91647ab1db898a7a67d6d3e7175d62466c69067f38d"} Jun 11 18:35:59.934292 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.934270 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 18:35:59.935202 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.935163 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientMemory" Jun 11 18:35:59.935283 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.935208 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 18:35:59.935283 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:35:59.935220 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeHasSufficientPID" Jun 11 18:36:00.006723 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:00.006676 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-156.ec2.internal\" not found" Jun 11 18:36:00.067422 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.067396 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 18:36:00.142989 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.142914 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal" Jun 11 18:36:00.156303 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.156283 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 18:36:00.157456 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.157442 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-156.ec2.internal" Jun 11 18:36:00.169333 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.169312 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 18:36:00.821154 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.821125 2561 apiserver.go:52] "Watching apiserver" Jun 11 18:36:00.828286 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.828255 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 11 18:36:00.829407 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.829370 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-58lbd","kube-system/konnectivity-agent-k8zpf","kube-system/kube-apiserver-proxy-ip-10-0-138-156.ec2.internal","openshift-dns/node-resolver-w4hqm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal","openshift-multus/multus-additional-cni-plugins-4qrvl","openshift-multus/multus-rzkwh","openshift-multus/network-metrics-daemon-fjvnc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5","openshift-cluster-node-tuning-operator/tuned-dsgqb","openshift-image-registry/node-ca-jx59l","openshift-network-diagnostics/network-check-target-dk499","openshift-network-operator/iptables-alerter-gmxgf","openshift-ovn-kubernetes/ovnkube-node-qhtcz"] Jun 11 18:36:00.831857 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.831841 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.833913 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.833883 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 11 18:36:00.833913 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.833892 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 11 18:36:00.834055 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.833914 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 11 18:36:00.834055 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.834043 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zl6j2\"" Jun 11 18:36:00.834152 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.834133 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jx59l" Jun 11 18:36:00.834299 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.834287 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 11 18:36:00.835934 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.835886 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 11 18:36:00.836632 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.836360 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-p8pjz\"" Jun 11 18:36:00.836632 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.836371 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 11 18:36:00.836632 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.836368 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 11 18:36:00.836632 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.836634 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.838557 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.838520 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 11 18:36:00.839424 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.838886 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 11 18:36:00.839424 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.839395 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w4hqm" Jun 11 18:36:00.840457 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.840438 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 11 18:36:00.840966 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.840792 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 11 18:36:00.840966 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.840943 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 11 18:36:00.841159 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.841137 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mp6bv\"" Jun 11 18:36:00.841291 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.841275 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 11 18:36:00.841754 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.841732 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 11 18:36:00.843068 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.842191 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9hmdm\"" Jun 11 18:36:00.843068 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.842262 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:00.843068 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:00.842398 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:00.843068 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.842457 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 11 18:36:00.843068 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.842570 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 11 18:36:00.845128 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.845112 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.846969 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.846949 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 11 18:36:00.847207 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.847188 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-k7kdr\"" Jun 11 18:36:00.847307 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.847221 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 11 18:36:00.847507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.847492 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:00.849461 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.849435 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 11 18:36:00.849729 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.849690 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 11 18:36:00.849729 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.849702 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 11 18:36:00.849880 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.849859 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x9g2h\"" Jun 11 18:36:00.850346 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.850328 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gmxgf" Jun 11 18:36:00.852251 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.852230 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 11 18:36:00.852377 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.852267 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ckvpn\"" Jun 11 18:36:00.852605 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.852588 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 11 18:36:00.852722 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.852707 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 11 18:36:00.853008 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.852992 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k8zpf" Jun 11 18:36:00.853868 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.853845 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-run-netns\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.853964 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.853876 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-var-lib-cni-multus\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.853964 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.853909 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-var-lib-kubelet\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.853964 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.853934 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-cnibin\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.854110 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.853960 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-run-multus-certs\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.854110 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.853990 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/597fee71-168a-4df0-b3c0-6d65f17a570f-ovn-node-metrics-cert\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.854110 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854032 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.854110 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854093 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-kubelet\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.854284 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854121 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/597fee71-168a-4df0-b3c0-6d65f17a570f-env-overrides\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.854284 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854145 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-os-release\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.854284 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854187 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-cni-binary-copy\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.854284 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854212 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.854284 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854236 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-slash\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.854284 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854260 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-cni-binary-copy\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.854520 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854287 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-multus-socket-dir-parent\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.854520 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854313 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf752\" (UniqueName: \"kubernetes.io/projected/ed57ef82-e664-4ffd-b72c-3fc0422eacab-kube-api-access-xf752\") pod \"node-ca-jx59l\" (UID: \"ed57ef82-e664-4ffd-b72c-3fc0422eacab\") " pod="openshift-image-registry/node-ca-jx59l" Jun 11 18:36:00.854520 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854339 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-cni-bin\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.854520 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854387 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/597fee71-168a-4df0-b3c0-6d65f17a570f-ovnkube-config\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.854520 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854411 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bac4fe8-0f72-4e10-8c74-3df40f3e6a05-tmp-dir\") pod \"node-resolver-w4hqm\" (UID: \"3bac4fe8-0f72-4e10-8c74-3df40f3e6a05\") " pod="openshift-dns/node-resolver-w4hqm" Jun 11 18:36:00.854759 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854560 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-multus-cni-dir\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.854759 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854591 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-var-lib-cni-bin\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.854759 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854611 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 18:36:00.854759 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854662 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed57ef82-e664-4ffd-b72c-3fc0422eacab-host\") pod \"node-ca-jx59l\" (UID: \"ed57ef82-e664-4ffd-b72c-3fc0422eacab\") " pod="openshift-image-registry/node-ca-jx59l" Jun 11 18:36:00.854943 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854769 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-systemd-units\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.854943 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854794 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-log-socket\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.854943 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854857 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-run-ovn-kubernetes\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.855079 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854927 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3bac4fe8-0f72-4e10-8c74-3df40f3e6a05-hosts-file\") pod \"node-resolver-w4hqm\" (UID: \"3bac4fe8-0f72-4e10-8c74-3df40f3e6a05\") " pod="openshift-dns/node-resolver-w4hqm" Jun 11 18:36:00.855079 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.854897 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 11 18:36:00.855079 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855036 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fq8wl\"" Jun 11 18:36:00.855079 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855060 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:00.855305 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855286 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 11 18:36:00.855616 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855595 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktm2r\" (UniqueName: \"kubernetes.io/projected/3bac4fe8-0f72-4e10-8c74-3df40f3e6a05-kube-api-access-ktm2r\") pod \"node-resolver-w4hqm\" (UID: \"3bac4fe8-0f72-4e10-8c74-3df40f3e6a05\") " pod="openshift-dns/node-resolver-w4hqm" Jun 11 18:36:00.855688 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855634 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-run-netns\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.855688 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855682 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-var-lib-openvswitch\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.855793 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855704 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-node-log\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.855793 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855745 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs\") pod \"network-metrics-daemon-fjvnc\" (UID: \"0a9668ae-c995-40a8-b7fd-de034a8fff15\") " pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:00.855793 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855770 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr4nk\" (UniqueName: \"kubernetes.io/projected/0a9668ae-c995-40a8-b7fd-de034a8fff15-kube-api-access-rr4nk\") pod \"network-metrics-daemon-fjvnc\" (UID: \"0a9668ae-c995-40a8-b7fd-de034a8fff15\") " pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:00.855940 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855795 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-system-cni-dir\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.855940 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855845 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-cnibin\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.855940 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855896 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-hostroot\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.855940 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855921 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-multus-conf-dir\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.855940 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855941 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-etc-kubernetes\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.856184 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855957 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-run-systemd\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.856184 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.855978 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-run-openvswitch\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.856184 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856003 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.856184 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856035 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/597fee71-168a-4df0-b3c0-6d65f17a570f-ovnkube-script-lib\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.856184 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856059 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-system-cni-dir\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.856184 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856101 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-os-release\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.856184 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856130 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbp58\" (UniqueName: \"kubernetes.io/projected/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-kube-api-access-cbp58\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.856184 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856165 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed57ef82-e664-4ffd-b72c-3fc0422eacab-serviceca\") pod \"node-ca-jx59l\" (UID: \"ed57ef82-e664-4ffd-b72c-3fc0422eacab\") " pod="openshift-image-registry/node-ca-jx59l" Jun 11 18:36:00.856529 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856222 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-etc-openvswitch\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.856529 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856245 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-run-ovn\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.856529 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856279 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sljt6\" (UniqueName: \"kubernetes.io/projected/597fee71-168a-4df0-b3c0-6d65f17a570f-kube-api-access-sljt6\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.856529 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856345 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.856529 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856367 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-run-k8s-cni-cncf-io\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.856529 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856392 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-multus-daemon-config\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.856529 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856442 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-cni-netd\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.856529 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856478 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b7xr\" (UniqueName: \"kubernetes.io/projected/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-kube-api-access-9b7xr\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.856927 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856913 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 11 18:36:00.856966 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.856947 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 11 18:36:00.857164 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.857149 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7rwn4\"" Jun 11 18:36:00.857672 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.857648 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:00.857827 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:00.857788 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:00.859988 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.859964 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:00.860074 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:00.860025 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:00.875353 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.875336 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gzqcv" Jun 11 18:36:00.882924 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.882904 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gzqcv" Jun 11 18:36:00.945056 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.945027 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 11 18:36:00.957288 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957252 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-run-openvswitch\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.957807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957298 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.957807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957354 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.957807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957358 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-run-openvswitch\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.957807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957405 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-system-cni-dir\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.957807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957445 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-etc-openvswitch\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.957807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957471 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-run-ovn\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.957807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957503 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/21a1280a-bea2-4fe2-8795-a55d4dc7c65b-agent-certs\") pod \"konnectivity-agent-k8zpf\" (UID: \"21a1280a-bea2-4fe2-8795-a55d4dc7c65b\") " pod="kube-system/konnectivity-agent-k8zpf" Jun 11 18:36:00.957807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957530 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-lib-modules\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:00.957807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957535 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-etc-openvswitch\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.957807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957554 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-cni-netd\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.957807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957504 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-system-cni-dir\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.957807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957581 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b7xr\" (UniqueName: \"kubernetes.io/projected/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-kube-api-access-9b7xr\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.957807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957617 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-run-ovn\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.957807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957669 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-var-lib-kubelet\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:00.957807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957787 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-cnibin\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957839 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/597fee71-168a-4df0-b3c0-6d65f17a570f-ovn-node-metrics-cert\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957865 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957893 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/21a1280a-bea2-4fe2-8795-a55d4dc7c65b-konnectivity-ca\") pod \"konnectivity-agent-k8zpf\" (UID: \"21a1280a-bea2-4fe2-8795-a55d4dc7c65b\") " pod="kube-system/konnectivity-agent-k8zpf" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957951 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-tuned\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.957977 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958002 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958024 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-socket-dir\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958048 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-device-dir\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958110 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-modprobe-d\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958134 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-host\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958157 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb731d8a-734b-48f1-87a2-303368226673-kubelet-config\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958202 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-slash\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958231 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-multus-socket-dir-parent\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958255 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bac4fe8-0f72-4e10-8c74-3df40f3e6a05-tmp-dir\") pod \"node-resolver-w4hqm\" (UID: \"3bac4fe8-0f72-4e10-8c74-3df40f3e6a05\") " pod="openshift-dns/node-resolver-w4hqm" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958285 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wtf7\" (UniqueName: \"kubernetes.io/projected/ace21358-d613-4e5e-bd91-a9112b770eb1-kube-api-access-8wtf7\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:00.958459 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958309 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-kubernetes\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958331 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-sysctl-conf\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958417 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-systemd\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958443 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-systemd-units\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958491 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-run-ovn-kubernetes\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958538 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktm2r\" (UniqueName: \"kubernetes.io/projected/3bac4fe8-0f72-4e10-8c74-3df40f3e6a05-kube-api-access-ktm2r\") pod \"node-resolver-w4hqm\" (UID: \"3bac4fe8-0f72-4e10-8c74-3df40f3e6a05\") " pod="openshift-dns/node-resolver-w4hqm" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958546 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958582 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958608 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frv7t\" (UniqueName: \"kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t\") pod \"network-check-target-dk499\" (UID: \"67b90c12-c8b0-4ceb-a8df-8369f7fff5a0\") " pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958626 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-cni-netd\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958647 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-node-log\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958672 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs\") pod \"network-metrics-daemon-fjvnc\" (UID: \"0a9668ae-c995-40a8-b7fd-de034a8fff15\") " pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958703 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-cnibin\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958730 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-hostroot\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958771 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-multus-conf-dir\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958862 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-cnibin\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958890 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-cnibin\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.959355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958902 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-run-ovn-kubernetes\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958914 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-run-systemd\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958951 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/597fee71-168a-4df0-b3c0-6d65f17a570f-ovnkube-script-lib\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958960 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958980 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-multus-conf-dir\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958980 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73d120d3-865a-4efd-ba23-0c18305c837c-host-slash\") pod \"iptables-alerter-gmxgf\" (UID: \"73d120d3-865a-4efd-ba23-0c18305c837c\") " pod="openshift-network-operator/iptables-alerter-gmxgf" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959015 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6md8\" (UniqueName: \"kubernetes.io/projected/73d120d3-865a-4efd-ba23-0c18305c837c-kube-api-access-q6md8\") pod \"iptables-alerter-gmxgf\" (UID: \"73d120d3-865a-4efd-ba23-0c18305c837c\") " pod="openshift-network-operator/iptables-alerter-gmxgf" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959044 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-os-release\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959067 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbp58\" (UniqueName: \"kubernetes.io/projected/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-kube-api-access-cbp58\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959091 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed57ef82-e664-4ffd-b72c-3fc0422eacab-serviceca\") pod \"node-ca-jx59l\" (UID: \"ed57ef82-e664-4ffd-b72c-3fc0422eacab\") " pod="openshift-image-registry/node-ca-jx59l" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959115 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sljt6\" (UniqueName: \"kubernetes.io/projected/597fee71-168a-4df0-b3c0-6d65f17a570f-kube-api-access-sljt6\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959139 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-etc-selinux\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959157 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bac4fe8-0f72-4e10-8c74-3df40f3e6a05-tmp-dir\") pod \"node-resolver-w4hqm\" (UID: \"3bac4fe8-0f72-4e10-8c74-3df40f3e6a05\") " pod="openshift-dns/node-resolver-w4hqm" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959163 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/73d120d3-865a-4efd-ba23-0c18305c837c-iptables-alerter-script\") pod \"iptables-alerter-gmxgf\" (UID: \"73d120d3-865a-4efd-ba23-0c18305c837c\") " pod="openshift-network-operator/iptables-alerter-gmxgf" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959217 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-systemd-units\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.958952 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-multus-socket-dir-parent\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959275 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f30b39f8-291b-4f26-9aad-7d29eabf73f7-tmp\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959296 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-slash\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.960200 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959313 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-os-release\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959342 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-node-log\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:00.959411 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:00.959462 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs podName:0a9668ae-c995-40a8-b7fd-de034a8fff15 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:01.459445711 +0000 UTC m=+3.027519176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs") pod "network-metrics-daemon-fjvnc" (UID: "0a9668ae-c995-40a8-b7fd-de034a8fff15") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959582 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-hostroot\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959629 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-run-systemd\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959650 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959300 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb731d8a-734b-48f1-87a2-303368226673-dbus\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959709 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-run-k8s-cni-cncf-io\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959757 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-run-k8s-cni-cncf-io\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959773 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-multus-daemon-config\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959811 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-run\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959786 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed57ef82-e664-4ffd-b72c-3fc0422eacab-serviceca\") pod \"node-ca-jx59l\" (UID: \"ed57ef82-e664-4ffd-b72c-3fc0422eacab\") " pod="openshift-image-registry/node-ca-jx59l" Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959841 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-run-netns\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959869 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-var-lib-cni-multus\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959888 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-run-netns\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959921 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-var-lib-kubelet\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.961126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959928 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-var-lib-cni-multus\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959956 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-run-multus-certs\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.959980 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-kubelet\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960019 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-run-multus-certs\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960005 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/597fee71-168a-4df0-b3c0-6d65f17a570f-env-overrides\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960055 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-var-lib-kubelet\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960068 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-os-release\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960091 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-kubelet\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960095 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-cni-binary-copy\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960090 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/597fee71-168a-4df0-b3c0-6d65f17a570f-ovnkube-script-lib\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960121 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-sys\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960132 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-multus-daemon-config\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960253 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-cni-binary-copy\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960263 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-os-release\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960283 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf752\" (UniqueName: \"kubernetes.io/projected/ed57ef82-e664-4ffd-b72c-3fc0422eacab-kube-api-access-xf752\") pod \"node-ca-jx59l\" (UID: \"ed57ef82-e664-4ffd-b72c-3fc0422eacab\") " pod="openshift-image-registry/node-ca-jx59l" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960321 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-cni-bin\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960346 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/597fee71-168a-4df0-b3c0-6d65f17a570f-ovnkube-config\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.961965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960386 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-cni-bin\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960417 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-sysconfig\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960433 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/597fee71-168a-4df0-b3c0-6d65f17a570f-env-overrides\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960440 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-sysctl-d\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960469 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-multus-cni-dir\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960494 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-var-lib-cni-bin\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960517 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed57ef82-e664-4ffd-b72c-3fc0422eacab-host\") pod \"node-ca-jx59l\" (UID: \"ed57ef82-e664-4ffd-b72c-3fc0422eacab\") " pod="openshift-image-registry/node-ca-jx59l" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960539 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-log-socket\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960580 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3bac4fe8-0f72-4e10-8c74-3df40f3e6a05-hosts-file\") pod \"node-resolver-w4hqm\" (UID: \"3bac4fe8-0f72-4e10-8c74-3df40f3e6a05\") " pod="openshift-dns/node-resolver-w4hqm" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960606 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr4nk\" (UniqueName: \"kubernetes.io/projected/0a9668ae-c995-40a8-b7fd-de034a8fff15-kube-api-access-rr4nk\") pod \"network-metrics-daemon-fjvnc\" (UID: \"0a9668ae-c995-40a8-b7fd-de034a8fff15\") " pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960630 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960656 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znbw7\" (UniqueName: \"kubernetes.io/projected/f30b39f8-291b-4f26-9aad-7d29eabf73f7-kube-api-access-znbw7\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960683 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-run-netns\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960707 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-var-lib-openvswitch\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960715 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-cni-binary-copy\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960758 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-system-cni-dir\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960785 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-registration-dir\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:00.962829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960833 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-sys-fs\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:00.963763 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960860 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-etc-kubernetes\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.963763 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.960966 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-etc-kubernetes\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.963763 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.961054 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-var-lib-openvswitch\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.963763 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.961075 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/597fee71-168a-4df0-b3c0-6d65f17a570f-ovnkube-config\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.963763 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.961100 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-log-socket\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.963763 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.961102 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.963763 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.961136 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-cni-binary-copy\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.963763 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.961136 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-system-cni-dir\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.963763 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.961189 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-host-var-lib-cni-bin\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.963763 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.961194 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-multus-cni-dir\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.963763 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.961203 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/597fee71-168a-4df0-b3c0-6d65f17a570f-host-run-netns\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.963763 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.961212 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed57ef82-e664-4ffd-b72c-3fc0422eacab-host\") pod \"node-ca-jx59l\" (UID: \"ed57ef82-e664-4ffd-b72c-3fc0422eacab\") " pod="openshift-image-registry/node-ca-jx59l" Jun 11 18:36:00.963763 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.961232 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3bac4fe8-0f72-4e10-8c74-3df40f3e6a05-hosts-file\") pod \"node-resolver-w4hqm\" (UID: \"3bac4fe8-0f72-4e10-8c74-3df40f3e6a05\") " pod="openshift-dns/node-resolver-w4hqm" Jun 11 18:36:00.963763 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.962711 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/597fee71-168a-4df0-b3c0-6d65f17a570f-ovn-node-metrics-cert\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:00.968370 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.968345 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b7xr\" (UniqueName: \"kubernetes.io/projected/bb4565b0-e59b-448c-ab3d-f83fd88e8a62-kube-api-access-9b7xr\") pod \"multus-additional-cni-plugins-4qrvl\" (UID: \"bb4565b0-e59b-448c-ab3d-f83fd88e8a62\") " pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:00.969366 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.969342 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr4nk\" (UniqueName: \"kubernetes.io/projected/0a9668ae-c995-40a8-b7fd-de034a8fff15-kube-api-access-rr4nk\") pod \"network-metrics-daemon-fjvnc\" (UID: \"0a9668ae-c995-40a8-b7fd-de034a8fff15\") " pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:00.969805 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.969786 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf752\" (UniqueName: \"kubernetes.io/projected/ed57ef82-e664-4ffd-b72c-3fc0422eacab-kube-api-access-xf752\") pod \"node-ca-jx59l\" (UID: \"ed57ef82-e664-4ffd-b72c-3fc0422eacab\") " pod="openshift-image-registry/node-ca-jx59l" Jun 11 18:36:00.969854 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.969807 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbp58\" (UniqueName: \"kubernetes.io/projected/96a4084a-9e99-433f-b7e0-e49fdc7f8ead-kube-api-access-cbp58\") pod \"multus-rzkwh\" (UID: \"96a4084a-9e99-433f-b7e0-e49fdc7f8ead\") " pod="openshift-multus/multus-rzkwh" Jun 11 18:36:00.969854 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.969841 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktm2r\" (UniqueName: \"kubernetes.io/projected/3bac4fe8-0f72-4e10-8c74-3df40f3e6a05-kube-api-access-ktm2r\") pod \"node-resolver-w4hqm\" (UID: \"3bac4fe8-0f72-4e10-8c74-3df40f3e6a05\") " pod="openshift-dns/node-resolver-w4hqm" Jun 11 18:36:00.970318 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:00.970298 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sljt6\" (UniqueName: \"kubernetes.io/projected/597fee71-168a-4df0-b3c0-6d65f17a570f-kube-api-access-sljt6\") pod \"ovnkube-node-qhtcz\" (UID: \"597fee71-168a-4df0-b3c0-6d65f17a570f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:01.061943 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.061908 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-systemd\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.062128 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.061953 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:01.062128 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.061983 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frv7t\" (UniqueName: \"kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t\") pod \"network-check-target-dk499\" (UID: \"67b90c12-c8b0-4ceb-a8df-8369f7fff5a0\") " pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:01.062128 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062050 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73d120d3-865a-4efd-ba23-0c18305c837c-host-slash\") pod \"iptables-alerter-gmxgf\" (UID: \"73d120d3-865a-4efd-ba23-0c18305c837c\") " pod="openshift-network-operator/iptables-alerter-gmxgf" Jun 11 18:36:01.062128 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062074 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-systemd\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.062128 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062079 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6md8\" (UniqueName: \"kubernetes.io/projected/73d120d3-865a-4efd-ba23-0c18305c837c-kube-api-access-q6md8\") pod \"iptables-alerter-gmxgf\" (UID: \"73d120d3-865a-4efd-ba23-0c18305c837c\") " pod="openshift-network-operator/iptables-alerter-gmxgf" Jun 11 18:36:01.062128 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:01.062116 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 18:36:01.062128 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062124 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-etc-selinux\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062138 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73d120d3-865a-4efd-ba23-0c18305c837c-host-slash\") pod \"iptables-alerter-gmxgf\" (UID: \"73d120d3-865a-4efd-ba23-0c18305c837c\") " pod="openshift-network-operator/iptables-alerter-gmxgf" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062161 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/73d120d3-865a-4efd-ba23-0c18305c837c-iptables-alerter-script\") pod \"iptables-alerter-gmxgf\" (UID: \"73d120d3-865a-4efd-ba23-0c18305c837c\") " pod="openshift-network-operator/iptables-alerter-gmxgf" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:01.062193 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret podName:fb731d8a-734b-48f1-87a2-303368226673 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:01.5621604 +0000 UTC m=+3.130233871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret") pod "global-pull-secret-syncer-58lbd" (UID: "fb731d8a-734b-48f1-87a2-303368226673") : object "kube-system"/"original-pull-secret" not registered Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062219 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f30b39f8-291b-4f26-9aad-7d29eabf73f7-tmp\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062242 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb731d8a-734b-48f1-87a2-303368226673-dbus\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062267 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-run\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062300 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-sys\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062324 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-sysconfig\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062334 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-etc-selinux\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062345 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-sysctl-d\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062393 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-run\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062412 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znbw7\" (UniqueName: \"kubernetes.io/projected/f30b39f8-291b-4f26-9aad-7d29eabf73f7-kube-api-access-znbw7\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062418 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb731d8a-734b-48f1-87a2-303368226673-dbus\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062449 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-sys\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062460 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-sysctl-d\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062454 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-registration-dir\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:01.062507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062500 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-sysconfig\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062513 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-sys-fs\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062546 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/21a1280a-bea2-4fe2-8795-a55d4dc7c65b-agent-certs\") pod \"konnectivity-agent-k8zpf\" (UID: \"21a1280a-bea2-4fe2-8795-a55d4dc7c65b\") " pod="kube-system/konnectivity-agent-k8zpf" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062551 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-registration-dir\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062570 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-lib-modules\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062602 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-var-lib-kubelet\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062643 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/21a1280a-bea2-4fe2-8795-a55d4dc7c65b-konnectivity-ca\") pod \"konnectivity-agent-k8zpf\" (UID: \"21a1280a-bea2-4fe2-8795-a55d4dc7c65b\") " pod="kube-system/konnectivity-agent-k8zpf" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062670 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-tuned\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062696 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062706 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-lib-modules\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062721 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-socket-dir\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062602 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-sys-fs\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062747 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-device-dir\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062761 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-var-lib-kubelet\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062771 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-modprobe-d\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062793 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062796 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-host\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.063375 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062837 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb731d8a-734b-48f1-87a2-303368226673-kubelet-config\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:01.064106 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062842 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/73d120d3-865a-4efd-ba23-0c18305c837c-iptables-alerter-script\") pod \"iptables-alerter-gmxgf\" (UID: \"73d120d3-865a-4efd-ba23-0c18305c837c\") " pod="openshift-network-operator/iptables-alerter-gmxgf" Jun 11 18:36:01.064106 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062856 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-device-dir\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:01.064106 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.062887 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb731d8a-734b-48f1-87a2-303368226673-kubelet-config\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:01.064106 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.063382 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wtf7\" (UniqueName: \"kubernetes.io/projected/ace21358-d613-4e5e-bd91-a9112b770eb1-kube-api-access-8wtf7\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:01.064315 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.064120 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-kubernetes\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.064315 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.064149 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-sysctl-conf\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.064315 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.064044 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-modprobe-d\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.064315 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.063940 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ace21358-d613-4e5e-bd91-a9112b770eb1-socket-dir\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:01.064315 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.064294 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-kubernetes\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.064315 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.063985 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/21a1280a-bea2-4fe2-8795-a55d4dc7c65b-konnectivity-ca\") pod \"konnectivity-agent-k8zpf\" (UID: \"21a1280a-bea2-4fe2-8795-a55d4dc7c65b\") " pod="kube-system/konnectivity-agent-k8zpf" Jun 11 18:36:01.064575 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.064312 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-host\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.064575 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.064457 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-sysctl-conf\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.067079 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.065113 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f30b39f8-291b-4f26-9aad-7d29eabf73f7-tmp\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.067079 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.066263 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/21a1280a-bea2-4fe2-8795-a55d4dc7c65b-agent-certs\") pod \"konnectivity-agent-k8zpf\" (UID: \"21a1280a-bea2-4fe2-8795-a55d4dc7c65b\") " pod="kube-system/konnectivity-agent-k8zpf" Jun 11 18:36:01.068447 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.068388 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f30b39f8-291b-4f26-9aad-7d29eabf73f7-etc-tuned\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.068730 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:01.068708 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 18:36:01.068730 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:01.068729 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 18:36:01.068882 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:01.068742 2561 projected.go:196] Error preparing data for projected volume kube-api-access-frv7t for pod openshift-network-diagnostics/network-check-target-dk499: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 18:36:01.068882 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:01.068824 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t podName:67b90c12-c8b0-4ceb-a8df-8369f7fff5a0 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:01.568806794 +0000 UTC m=+3.136880280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-frv7t" (UniqueName: "kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t") pod "network-check-target-dk499" (UID: "67b90c12-c8b0-4ceb-a8df-8369f7fff5a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 18:36:01.070934 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.070906 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6md8\" (UniqueName: \"kubernetes.io/projected/73d120d3-865a-4efd-ba23-0c18305c837c-kube-api-access-q6md8\") pod \"iptables-alerter-gmxgf\" (UID: \"73d120d3-865a-4efd-ba23-0c18305c837c\") " pod="openshift-network-operator/iptables-alerter-gmxgf" Jun 11 18:36:01.071033 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.070911 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znbw7\" (UniqueName: \"kubernetes.io/projected/f30b39f8-291b-4f26-9aad-7d29eabf73f7-kube-api-access-znbw7\") pod \"tuned-dsgqb\" (UID: \"f30b39f8-291b-4f26-9aad-7d29eabf73f7\") " pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.071843 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.071796 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wtf7\" (UniqueName: \"kubernetes.io/projected/ace21358-d613-4e5e-bd91-a9112b770eb1-kube-api-access-8wtf7\") pod \"aws-ebs-csi-driver-node-pq2x5\" (UID: \"ace21358-d613-4e5e-bd91-a9112b770eb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:01.145010 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.144972 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rzkwh" Jun 11 18:36:01.152053 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.152028 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jx59l" Jun 11 18:36:01.159881 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.159862 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:01.166350 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.166323 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w4hqm" Jun 11 18:36:01.174115 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.174091 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4qrvl" Jun 11 18:36:01.180814 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.180795 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" Jun 11 18:36:01.188435 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.188415 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gmxgf" Jun 11 18:36:01.199054 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.199028 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k8zpf" Jun 11 18:36:01.204715 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.204692 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" Jun 11 18:36:01.335352 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.335272 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 18:36:01.467078 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.467049 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs\") pod \"network-metrics-daemon-fjvnc\" (UID: \"0a9668ae-c995-40a8-b7fd-de034a8fff15\") " pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:01.467237 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:01.467187 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 18:36:01.467282 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:01.467240 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs podName:0a9668ae-c995-40a8-b7fd-de034a8fff15 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:02.467226793 +0000 UTC m=+4.035300251 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs") pod "network-metrics-daemon-fjvnc" (UID: "0a9668ae-c995-40a8-b7fd-de034a8fff15") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 18:36:01.567773 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.567738 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:01.567925 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:01.567901 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 18:36:01.567968 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:01.567962 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret podName:fb731d8a-734b-48f1-87a2-303368226673 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:02.567947282 +0000 UTC m=+4.136020740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret") pod "global-pull-secret-syncer-58lbd" (UID: "fb731d8a-734b-48f1-87a2-303368226673") : object "kube-system"/"original-pull-secret" not registered Jun 11 18:36:01.668524 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.668440 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frv7t\" (UniqueName: \"kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t\") pod \"network-check-target-dk499\" (UID: \"67b90c12-c8b0-4ceb-a8df-8369f7fff5a0\") " pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:01.668658 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:01.668614 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 18:36:01.668658 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:01.668639 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 18:36:01.668658 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:01.668649 2561 projected.go:196] Error preparing data for projected volume kube-api-access-frv7t for pod openshift-network-diagnostics/network-check-target-dk499: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 18:36:01.668768 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:01.668703 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t podName:67b90c12-c8b0-4ceb-a8df-8369f7fff5a0 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:02.668688923 +0000 UTC m=+4.236762381 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-frv7t" (UniqueName: "kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t") pod "network-check-target-dk499" (UID: "67b90c12-c8b0-4ceb-a8df-8369f7fff5a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 18:36:01.868550 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.868524 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 18:36:01.886275 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.886214 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 18:31:00 +0000 UTC" deadline="2028-01-11 03:54:57.057083897 +0000 UTC" Jun 11 18:36:01.886275 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:01.886270 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13881h18m55.170816289s" Jun 11 18:36:01.950577 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:01.950543 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bac4fe8_0f72_4e10_8c74_3df40f3e6a05.slice/crio-594a55d46aa7d7c3f78a1728df21169da4e17cfc9508b84d9e8e6101d7ab3bf2 WatchSource:0}: Error finding container 594a55d46aa7d7c3f78a1728df21169da4e17cfc9508b84d9e8e6101d7ab3bf2: Status 404 returned error can't find the container with id 594a55d46aa7d7c3f78a1728df21169da4e17cfc9508b84d9e8e6101d7ab3bf2 Jun 11 18:36:01.951878 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:01.951844 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod597fee71_168a_4df0_b3c0_6d65f17a570f.slice/crio-c9e11952318bece39f77c53880dc9af007818916bc46e9e2735130c2ec399df4 WatchSource:0}: Error finding container c9e11952318bece39f77c53880dc9af007818916bc46e9e2735130c2ec399df4: Status 404 returned error can't find the container with id c9e11952318bece39f77c53880dc9af007818916bc46e9e2735130c2ec399df4 Jun 11 18:36:01.953096 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:01.952934 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a4084a_9e99_433f_b7e0_e49fdc7f8ead.slice/crio-f083ec6638f7125eaf87ee324f99e21079a4d2075498788ab0ce0a7bb2558a6a WatchSource:0}: Error finding container f083ec6638f7125eaf87ee324f99e21079a4d2075498788ab0ce0a7bb2558a6a: Status 404 returned error can't find the container with id f083ec6638f7125eaf87ee324f99e21079a4d2075498788ab0ce0a7bb2558a6a Jun 11 18:36:01.955516 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:01.955490 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace21358_d613_4e5e_bd91_a9112b770eb1.slice/crio-6b1bc5b8772d3881f58e02f82a0c73b9a847bf17b2c41b0c93318ff5c5e771fc WatchSource:0}: Error finding container 6b1bc5b8772d3881f58e02f82a0c73b9a847bf17b2c41b0c93318ff5c5e771fc: Status 404 returned error can't find the container with id 6b1bc5b8772d3881f58e02f82a0c73b9a847bf17b2c41b0c93318ff5c5e771fc Jun 11 18:36:01.956885 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:01.956516 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d120d3_865a_4efd_ba23_0c18305c837c.slice/crio-83d9c7bbe2e25dbbb04857be63fefb1612302d0e02bc9586520ee827aa2576f5 WatchSource:0}: Error finding container 83d9c7bbe2e25dbbb04857be63fefb1612302d0e02bc9586520ee827aa2576f5: Status 404 returned error can't find the container with id 83d9c7bbe2e25dbbb04857be63fefb1612302d0e02bc9586520ee827aa2576f5 Jun 11 18:36:01.957277 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:01.957199 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21a1280a_bea2_4fe2_8795_a55d4dc7c65b.slice/crio-9cabb762c2cba218f24d46eecbd16342efa674555835090549d5bf1c63a1fc11 WatchSource:0}: Error finding container 9cabb762c2cba218f24d46eecbd16342efa674555835090549d5bf1c63a1fc11: Status 404 returned error can't find the container with id 9cabb762c2cba218f24d46eecbd16342efa674555835090549d5bf1c63a1fc11 Jun 11 18:36:01.958469 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:01.958267 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf30b39f8_291b_4f26_9aad_7d29eabf73f7.slice/crio-7ad44b810a431f82a8001fb019573866b2a9ea3d31d199c33e4e118bb8405400 WatchSource:0}: Error finding container 7ad44b810a431f82a8001fb019573866b2a9ea3d31d199c33e4e118bb8405400: Status 404 returned error can't find the container with id 7ad44b810a431f82a8001fb019573866b2a9ea3d31d199c33e4e118bb8405400 Jun 11 18:36:01.959617 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:01.959592 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb4565b0_e59b_448c_ab3d_f83fd88e8a62.slice/crio-e60e2d7ffcfc277494a2761756181535aba7c8d5682c8093f6da148fb0b9916f WatchSource:0}: Error finding container e60e2d7ffcfc277494a2761756181535aba7c8d5682c8093f6da148fb0b9916f: Status 404 returned error can't find the container with id e60e2d7ffcfc277494a2761756181535aba7c8d5682c8093f6da148fb0b9916f Jun 11 18:36:01.961329 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:01.961261 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded57ef82_e664_4ffd_b72c_3fc0422eacab.slice/crio-65de9bc1260d6bcc7b614adb10dea6422d049320070460e99f3894394ff89d5c WatchSource:0}: Error finding container 65de9bc1260d6bcc7b614adb10dea6422d049320070460e99f3894394ff89d5c: Status 404 returned error can't find the container with id 65de9bc1260d6bcc7b614adb10dea6422d049320070460e99f3894394ff89d5c Jun 11 18:36:02.474358 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:02.473508 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs\") pod \"network-metrics-daemon-fjvnc\" (UID: \"0a9668ae-c995-40a8-b7fd-de034a8fff15\") " pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:02.474358 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:02.473874 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 18:36:02.474358 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:02.473941 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs podName:0a9668ae-c995-40a8-b7fd-de034a8fff15 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:04.473919246 +0000 UTC m=+6.041992705 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs") pod "network-metrics-daemon-fjvnc" (UID: "0a9668ae-c995-40a8-b7fd-de034a8fff15") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 18:36:02.575386 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:02.574666 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:02.575386 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:02.574841 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 18:36:02.575386 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:02.574921 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret podName:fb731d8a-734b-48f1-87a2-303368226673 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:04.574893059 +0000 UTC m=+6.142966531 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret") pod "global-pull-secret-syncer-58lbd" (UID: "fb731d8a-734b-48f1-87a2-303368226673") : object "kube-system"/"original-pull-secret" not registered Jun 11 18:36:02.675334 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:02.675284 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frv7t\" (UniqueName: \"kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t\") pod \"network-check-target-dk499\" (UID: \"67b90c12-c8b0-4ceb-a8df-8369f7fff5a0\") " pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:02.675538 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:02.675465 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 18:36:02.675538 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:02.675490 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 18:36:02.675538 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:02.675503 2561 projected.go:196] Error preparing data for projected volume kube-api-access-frv7t for pod openshift-network-diagnostics/network-check-target-dk499: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 18:36:02.675707 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:02.675561 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t podName:67b90c12-c8b0-4ceb-a8df-8369f7fff5a0 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:04.675542753 +0000 UTC m=+6.243616228 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-frv7t" (UniqueName: "kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t") pod "network-check-target-dk499" (UID: "67b90c12-c8b0-4ceb-a8df-8369f7fff5a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 18:36:02.887405 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:02.887267 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 18:31:00 +0000 UTC" deadline="2027-11-27 18:15:51.067749757 +0000 UTC" Jun 11 18:36:02.887405 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:02.887306 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12815h39m48.180448275s" Jun 11 18:36:02.915754 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:02.914811 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:02.915754 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:02.914954 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:02.915754 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:02.915397 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:02.915754 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:02.915497 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:02.915754 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:02.915590 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:02.915754 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:02.915681 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:02.929407 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:02.929322 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k8zpf" event={"ID":"21a1280a-bea2-4fe2-8795-a55d4dc7c65b","Type":"ContainerStarted","Data":"9cabb762c2cba218f24d46eecbd16342efa674555835090549d5bf1c63a1fc11"} Jun 11 18:36:02.954780 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:02.954708 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gmxgf" event={"ID":"73d120d3-865a-4efd-ba23-0c18305c837c","Type":"ContainerStarted","Data":"83d9c7bbe2e25dbbb04857be63fefb1612302d0e02bc9586520ee827aa2576f5"} Jun 11 18:36:02.962280 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:02.962148 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" event={"ID":"ace21358-d613-4e5e-bd91-a9112b770eb1","Type":"ContainerStarted","Data":"6b1bc5b8772d3881f58e02f82a0c73b9a847bf17b2c41b0c93318ff5c5e771fc"} Jun 11 18:36:02.972068 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:02.971370 2561 generic.go:356] "Generic (PLEG): container finished" podID="f345c025a1c78c555e7e71fabd161286" containerID="3ba9c53988b9bc8770591bdcc09532f148e86ec5e2d63eff6f35309086ac3d78" exitCode=0 Jun 11 18:36:02.972068 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:02.971756 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal" event={"ID":"f345c025a1c78c555e7e71fabd161286","Type":"ContainerDied","Data":"3ba9c53988b9bc8770591bdcc09532f148e86ec5e2d63eff6f35309086ac3d78"} Jun 11 18:36:02.980532 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:02.980445 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-156.ec2.internal" event={"ID":"536524fd725acf153e94591109a49596","Type":"ContainerStarted","Data":"bbdb412f3dc2d2aafeb671cbf631a50a214bdb6a598cfe4fbe6df6fc1f3d3c39"} Jun 11 18:36:02.994825 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:02.994745 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jx59l" event={"ID":"ed57ef82-e664-4ffd-b72c-3fc0422eacab","Type":"ContainerStarted","Data":"65de9bc1260d6bcc7b614adb10dea6422d049320070460e99f3894394ff89d5c"} Jun 11 18:36:03.003103 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:03.003033 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rzkwh" event={"ID":"96a4084a-9e99-433f-b7e0-e49fdc7f8ead","Type":"ContainerStarted","Data":"f083ec6638f7125eaf87ee324f99e21079a4d2075498788ab0ce0a7bb2558a6a"} Jun 11 18:36:03.005478 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:03.004357 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-156.ec2.internal" podStartSLOduration=3.004339042 podStartE2EDuration="3.004339042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 18:36:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:03.004249591 +0000 UTC m=+4.572323072" watchObservedRunningTime="2026-06-11 18:36:03.004339042 +0000 UTC m=+4.572412522" Jun 11 18:36:03.006002 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:03.005975 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" event={"ID":"597fee71-168a-4df0-b3c0-6d65f17a570f","Type":"ContainerStarted","Data":"c9e11952318bece39f77c53880dc9af007818916bc46e9e2735130c2ec399df4"} Jun 11 18:36:03.008881 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:03.008811 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w4hqm" event={"ID":"3bac4fe8-0f72-4e10-8c74-3df40f3e6a05","Type":"ContainerStarted","Data":"594a55d46aa7d7c3f78a1728df21169da4e17cfc9508b84d9e8e6101d7ab3bf2"} Jun 11 18:36:03.045076 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:03.045038 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4qrvl" event={"ID":"bb4565b0-e59b-448c-ab3d-f83fd88e8a62","Type":"ContainerStarted","Data":"e60e2d7ffcfc277494a2761756181535aba7c8d5682c8093f6da148fb0b9916f"} Jun 11 18:36:03.062845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:03.062740 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" event={"ID":"f30b39f8-291b-4f26-9aad-7d29eabf73f7","Type":"ContainerStarted","Data":"7ad44b810a431f82a8001fb019573866b2a9ea3d31d199c33e4e118bb8405400"} Jun 11 18:36:04.088760 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:04.088603 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal" event={"ID":"f345c025a1c78c555e7e71fabd161286","Type":"ContainerStarted","Data":"b458d06e418b3a9a118551f31e0d8f3d896db286abcc1de13f545131081a68c6"} Jun 11 18:36:04.104538 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:04.103715 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-156.ec2.internal" podStartSLOduration=4.103694092 podStartE2EDuration="4.103694092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 18:36:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:04.103477869 +0000 UTC m=+5.671551375" watchObservedRunningTime="2026-06-11 18:36:04.103694092 +0000 UTC m=+5.671767573" Jun 11 18:36:04.489759 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:04.489663 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs\") pod \"network-metrics-daemon-fjvnc\" (UID: \"0a9668ae-c995-40a8-b7fd-de034a8fff15\") " pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:04.489946 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:04.489853 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 18:36:04.489946 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:04.489916 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs podName:0a9668ae-c995-40a8-b7fd-de034a8fff15 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:08.489896205 +0000 UTC m=+10.057969668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs") pod "network-metrics-daemon-fjvnc" (UID: "0a9668ae-c995-40a8-b7fd-de034a8fff15") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 18:36:04.591295 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:04.590708 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:04.591295 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:04.590875 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 18:36:04.591295 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:04.590931 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret podName:fb731d8a-734b-48f1-87a2-303368226673 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:08.590915211 +0000 UTC m=+10.158988675 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret") pod "global-pull-secret-syncer-58lbd" (UID: "fb731d8a-734b-48f1-87a2-303368226673") : object "kube-system"/"original-pull-secret" not registered Jun 11 18:36:04.691451 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:04.691419 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frv7t\" (UniqueName: \"kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t\") pod \"network-check-target-dk499\" (UID: \"67b90c12-c8b0-4ceb-a8df-8369f7fff5a0\") " pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:04.691629 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:04.691576 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 18:36:04.691629 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:04.691596 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 18:36:04.691629 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:04.691608 2561 projected.go:196] Error preparing data for projected volume kube-api-access-frv7t for pod openshift-network-diagnostics/network-check-target-dk499: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 18:36:04.691779 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:04.691668 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t podName:67b90c12-c8b0-4ceb-a8df-8369f7fff5a0 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:08.691650299 +0000 UTC m=+10.259723778 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-frv7t" (UniqueName: "kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t") pod "network-check-target-dk499" (UID: "67b90c12-c8b0-4ceb-a8df-8369f7fff5a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 18:36:04.912911 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:04.912830 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:04.913070 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:04.912980 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:04.913409 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:04.913381 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:04.913507 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:04.913477 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:04.913556 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:04.913537 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:04.913655 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:04.913599 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:06.912708 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:06.912664 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:06.913162 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:06.912666 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:06.913162 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:06.912797 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:06.913162 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:06.912678 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:06.913162 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:06.912886 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:06.913162 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:06.912953 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:08.519078 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:08.519040 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs\") pod \"network-metrics-daemon-fjvnc\" (UID: \"0a9668ae-c995-40a8-b7fd-de034a8fff15\") " pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:08.519535 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:08.519244 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 18:36:08.519535 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:08.519323 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs podName:0a9668ae-c995-40a8-b7fd-de034a8fff15 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:16.519303963 +0000 UTC m=+18.087377425 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs") pod "network-metrics-daemon-fjvnc" (UID: "0a9668ae-c995-40a8-b7fd-de034a8fff15") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 18:36:08.619993 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:08.619956 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:08.620198 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:08.620099 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 18:36:08.620198 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:08.620167 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret podName:fb731d8a-734b-48f1-87a2-303368226673 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:16.620146764 +0000 UTC m=+18.188220238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret") pod "global-pull-secret-syncer-58lbd" (UID: "fb731d8a-734b-48f1-87a2-303368226673") : object "kube-system"/"original-pull-secret" not registered Jun 11 18:36:08.721400 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:08.721349 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frv7t\" (UniqueName: \"kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t\") pod \"network-check-target-dk499\" (UID: \"67b90c12-c8b0-4ceb-a8df-8369f7fff5a0\") " pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:08.721585 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:08.721551 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 18:36:08.721585 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:08.721575 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 18:36:08.721701 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:08.721590 2561 projected.go:196] Error preparing data for projected volume kube-api-access-frv7t for pod openshift-network-diagnostics/network-check-target-dk499: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 18:36:08.721701 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:08.721655 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t podName:67b90c12-c8b0-4ceb-a8df-8369f7fff5a0 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:16.721634771 +0000 UTC m=+18.289708229 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-frv7t" (UniqueName: "kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t") pod "network-check-target-dk499" (UID: "67b90c12-c8b0-4ceb-a8df-8369f7fff5a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 18:36:08.916339 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:08.916268 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:08.916492 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:08.916411 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:08.916492 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:08.916446 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:08.916607 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:08.916453 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:08.916607 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:08.916562 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:08.916698 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:08.916648 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:10.912542 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:10.912507 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:10.912996 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:10.912507 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:10.912996 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:10.912647 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:10.912996 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:10.912699 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:10.912996 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:10.912750 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:10.912996 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:10.912886 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:12.106463 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:12.106184 2561 generic.go:356] "Generic (PLEG): container finished" podID="bb4565b0-e59b-448c-ab3d-f83fd88e8a62" containerID="0fbc69678f88498c73547c102cef925082d23d658b5ad1148388312f34a7b1f3" exitCode=0 Jun 11 18:36:12.106896 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:12.106273 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4qrvl" event={"ID":"bb4565b0-e59b-448c-ab3d-f83fd88e8a62","Type":"ContainerDied","Data":"0fbc69678f88498c73547c102cef925082d23d658b5ad1148388312f34a7b1f3"} Jun 11 18:36:12.108626 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:12.108574 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" event={"ID":"f30b39f8-291b-4f26-9aad-7d29eabf73f7","Type":"ContainerStarted","Data":"82efdac7919ac4c9e7dd4a6fa38c8a0628991ceca9eea9c1a01e380715a9fc1a"} Jun 11 18:36:12.110219 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:12.110193 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k8zpf" event={"ID":"21a1280a-bea2-4fe2-8795-a55d4dc7c65b","Type":"ContainerStarted","Data":"30a0caca6a9a4bee50fdd8b019ff81eabd6e434dd39fc626fa597af7a1595bb3"} Jun 11 18:36:12.111825 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:12.111802 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" event={"ID":"ace21358-d613-4e5e-bd91-a9112b770eb1","Type":"ContainerStarted","Data":"6edffdd7e3af27e23f6d90386f7904af4951b2689f4ceedd561919efeaa7bad8"} Jun 11 18:36:12.113327 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:12.113304 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jx59l" event={"ID":"ed57ef82-e664-4ffd-b72c-3fc0422eacab","Type":"ContainerStarted","Data":"677ac9e9208aafd788d502d2942d93266cde15139b8e149bb1177b0cf631161d"} Jun 11 18:36:12.114547 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:12.114522 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w4hqm" event={"ID":"3bac4fe8-0f72-4e10-8c74-3df40f3e6a05","Type":"ContainerStarted","Data":"cf4231a7ddfcc7fcb62585feed48e2fa6fea62030f64e6a9506cf94f496968a5"} Jun 11 18:36:12.144047 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:12.143987 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jx59l" podStartSLOduration=3.6826286 podStartE2EDuration="13.143966393s" totalImagesPullingTime="9.461337793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 18:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:12.143189188 +0000 UTC m=+13.711262672" watchObservedRunningTime="2026-06-11 18:36:12.143966393 +0000 UTC m=+13.712039874" Jun 11 18:36:12.157770 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:12.157714 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-w4hqm" podStartSLOduration=3.712178781 podStartE2EDuration="13.157694532s" totalImagesPullingTime="9.445515751s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 18:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:12.157308061 +0000 UTC m=+13.725381542" watchObservedRunningTime="2026-06-11 18:36:12.157694532 +0000 UTC m=+13.725768013" Jun 11 18:36:12.172357 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:12.172273 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dsgqb" podStartSLOduration=3.67912666 podStartE2EDuration="13.172254947s" totalImagesPullingTime="9.493128287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 18:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:12.171997807 +0000 UTC m=+13.740071313" watchObservedRunningTime="2026-06-11 18:36:12.172254947 +0000 UTC m=+13.740328428" Jun 11 18:36:12.188142 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:12.188091 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-k8zpf" podStartSLOduration=3.753809666 podStartE2EDuration="13.188070507s" totalImagesPullingTime="9.434260841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:12.187760981 +0000 UTC m=+13.755834462" watchObservedRunningTime="2026-06-11 18:36:12.188070507 +0000 UTC m=+13.756143990" Jun 11 18:36:12.913307 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:12.913278 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:12.913432 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:12.913408 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:12.913432 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:12.913278 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:12.913559 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:12.913537 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:12.913614 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:12.913571 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:12.913671 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:12.913653 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:13.118609 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:13.118538 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gmxgf" event={"ID":"73d120d3-865a-4efd-ba23-0c18305c837c","Type":"ContainerStarted","Data":"677423cecc5f8e44fbfbbeaf78a72772f661a1eb7a09ddb6e164c026224ac987"} Jun 11 18:36:13.133999 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:13.133956 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-gmxgf" podStartSLOduration=4.694710188 podStartE2EDuration="14.13394132s" totalImagesPullingTime="9.439231132s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 18:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:13.133651968 +0000 UTC m=+14.701725449" watchObservedRunningTime="2026-06-11 18:36:13.13394132 +0000 UTC m=+14.702014800" Jun 11 18:36:14.913083 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:14.913023 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:14.913083 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:14.913054 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:14.913709 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:14.913158 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:14.913709 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:14.913285 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:14.913709 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:14.913376 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:14.913709 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:14.913459 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:15.145709 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:15.145603 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-k8zpf" Jun 11 18:36:15.146257 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:15.146236 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-k8zpf" Jun 11 18:36:16.587123 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:16.587087 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs\") pod \"network-metrics-daemon-fjvnc\" (UID: \"0a9668ae-c995-40a8-b7fd-de034a8fff15\") " pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:16.587860 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:16.587244 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 18:36:16.587860 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:16.587305 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs podName:0a9668ae-c995-40a8-b7fd-de034a8fff15 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:32.58728527 +0000 UTC m=+34.155358727 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs") pod "network-metrics-daemon-fjvnc" (UID: "0a9668ae-c995-40a8-b7fd-de034a8fff15") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 18:36:16.687799 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:16.687764 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:16.687980 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:16.687925 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 18:36:16.688052 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:16.688007 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret podName:fb731d8a-734b-48f1-87a2-303368226673 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:32.687983566 +0000 UTC m=+34.256057027 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret") pod "global-pull-secret-syncer-58lbd" (UID: "fb731d8a-734b-48f1-87a2-303368226673") : object "kube-system"/"original-pull-secret" not registered Jun 11 18:36:16.788488 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:16.788450 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frv7t\" (UniqueName: \"kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t\") pod \"network-check-target-dk499\" (UID: \"67b90c12-c8b0-4ceb-a8df-8369f7fff5a0\") " pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:16.788676 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:16.788589 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 18:36:16.788676 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:16.788605 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 18:36:16.788676 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:16.788614 2561 projected.go:196] Error preparing data for projected volume kube-api-access-frv7t for pod openshift-network-diagnostics/network-check-target-dk499: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 18:36:16.788676 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:16.788674 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t podName:67b90c12-c8b0-4ceb-a8df-8369f7fff5a0 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:32.788654118 +0000 UTC m=+34.356727600 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-frv7t" (UniqueName: "kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t") pod "network-check-target-dk499" (UID: "67b90c12-c8b0-4ceb-a8df-8369f7fff5a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 18:36:16.915425 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:16.915362 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:16.915425 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:16.915403 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:16.915598 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:16.915463 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:16.915598 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:16.915481 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:16.915598 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:16.915529 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:16.915701 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:16.915601 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:18.913054 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:18.913023 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:18.913599 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:18.913122 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:18.913599 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:18.913199 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:18.913599 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:18.913283 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:18.913599 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:18.913330 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:18.913599 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:18.913406 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:19.398575 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:19.398489 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-k8zpf" Jun 11 18:36:19.398731 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:19.398657 2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 11 18:36:19.399445 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:19.399414 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-k8zpf" Jun 11 18:36:20.913123 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:20.913088 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:20.913123 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:20.913115 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:20.913661 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:20.913247 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:20.913661 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:20.913392 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:20.913661 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:20.913470 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:20.913661 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:20.913540 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:22.912378 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:22.912235 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:22.912378 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:22.912235 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:22.912898 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:22.912488 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:22.912898 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:22.912483 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:22.912898 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:22.912568 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:22.912898 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:22.912620 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:23.722103 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:23.722070 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 11 18:36:23.934587 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:23.934510 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-11T18:36:23.722097344Z","UUID":"fe9bd320-604a-4628-8eaa-b6b5276bcab0","Handler":null,"Name":"","Endpoint":""} Jun 11 18:36:23.936014 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:23.935993 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 11 18:36:23.936102 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:23.936027 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 11 18:36:24.143103 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:24.142915 2561 generic.go:356] "Generic (PLEG): container finished" podID="bb4565b0-e59b-448c-ab3d-f83fd88e8a62" containerID="64ee075ff67f26168bc303a9dddf50216028aa5f8f8f9b4de78a454a378b246d" exitCode=0 Jun 11 18:36:24.143213 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:24.142992 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4qrvl" event={"ID":"bb4565b0-e59b-448c-ab3d-f83fd88e8a62","Type":"ContainerDied","Data":"64ee075ff67f26168bc303a9dddf50216028aa5f8f8f9b4de78a454a378b246d"} Jun 11 18:36:24.146332 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:24.146221 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" event={"ID":"ace21358-d613-4e5e-bd91-a9112b770eb1","Type":"ContainerStarted","Data":"e65b0136bd5716f1c5dd53bda8998535ec79a1ab6f02f90c2bb1487fab02bf80"} Jun 11 18:36:24.147781 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:24.147755 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rzkwh" event={"ID":"96a4084a-9e99-433f-b7e0-e49fdc7f8ead","Type":"ContainerStarted","Data":"ee3cbd588b9c2388a01a6238ab96e33aefe62d42571508100d20996f9b5d39cf"} Jun 11 18:36:24.150481 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:24.150463 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 18:36:24.150840 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:24.150811 2561 generic.go:356] "Generic (PLEG): container finished" podID="597fee71-168a-4df0-b3c0-6d65f17a570f" containerID="ffe360825040069b204c6cd6b02e32133897b2ee946a948de1cf576448b6082a" exitCode=1 Jun 11 18:36:24.150914 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:24.150853 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" event={"ID":"597fee71-168a-4df0-b3c0-6d65f17a570f","Type":"ContainerStarted","Data":"33ab980111b82950147337781dfe5564279fb126a2edc4a9962becad0aea3ad8"} Jun 11 18:36:24.150953 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:24.150918 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" event={"ID":"597fee71-168a-4df0-b3c0-6d65f17a570f","Type":"ContainerStarted","Data":"f1f911fe1b793a24f5f1efaf380a9de2e4642e7819b6b69d51d8b5a76eeea822"} Jun 11 18:36:24.150985 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:24.150969 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" event={"ID":"597fee71-168a-4df0-b3c0-6d65f17a570f","Type":"ContainerStarted","Data":"7ae21985f9a4304615b454fd9c621fb48718602baf43831f6aae62c010cc6134"} Jun 11 18:36:24.151031 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:24.151014 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" event={"ID":"597fee71-168a-4df0-b3c0-6d65f17a570f","Type":"ContainerDied","Data":"ffe360825040069b204c6cd6b02e32133897b2ee946a948de1cf576448b6082a"} Jun 11 18:36:24.151138 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:24.151070 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" event={"ID":"597fee71-168a-4df0-b3c0-6d65f17a570f","Type":"ContainerStarted","Data":"e5db903b771221cf8ea72d959a618f8860cb449706a857123c78ded075d746fe"} Jun 11 18:36:24.185865 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:24.185797 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rzkwh" podStartSLOduration=3.6230350700000002 podStartE2EDuration="25.185779163s" totalImagesPullingTime="21.562744093s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 18:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:24.185608343 +0000 UTC m=+25.753681824" watchObservedRunningTime="2026-06-11 18:36:24.185779163 +0000 UTC m=+25.753852642" Jun 11 18:36:24.913167 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:24.913132 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:24.913167 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:24.913132 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:24.913412 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:24.913257 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:24.913412 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:24.913276 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:24.913412 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:24.913361 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:24.913570 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:24.913464 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:25.155009 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:25.154978 2561 generic.go:356] "Generic (PLEG): container finished" podID="bb4565b0-e59b-448c-ab3d-f83fd88e8a62" containerID="82cf04bf19f53fc48c49e63eb5c7b9d69427a3fffca6e7bf3ea640380b6d2ea6" exitCode=0 Jun 11 18:36:25.155533 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:25.155058 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4qrvl" event={"ID":"bb4565b0-e59b-448c-ab3d-f83fd88e8a62","Type":"ContainerDied","Data":"82cf04bf19f53fc48c49e63eb5c7b9d69427a3fffca6e7bf3ea640380b6d2ea6"} Jun 11 18:36:25.157224 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:25.157199 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" event={"ID":"ace21358-d613-4e5e-bd91-a9112b770eb1","Type":"ContainerStarted","Data":"d8b4694c0ec5c137281f940817993b162ccf590665d47702fbc8aeded1ab0a62"} Jun 11 18:36:25.159993 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:25.159976 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 18:36:25.160331 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:25.160314 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" event={"ID":"597fee71-168a-4df0-b3c0-6d65f17a570f","Type":"ContainerStarted","Data":"d9726f058ca5e7ea2043c0cf73984f57bd2fd04acfe36da596dffe736c1a0119"} Jun 11 18:36:25.194250 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:25.194204 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pq2x5" podStartSLOduration=3.891302937 podStartE2EDuration="26.194169102s" totalImagesPullingTime="22.302866165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 18:35:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:25.194085291 +0000 UTC m=+26.762158771" watchObservedRunningTime="2026-06-11 18:36:25.194169102 +0000 UTC m=+26.762259856" Jun 11 18:36:26.164759 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:26.164716 2561 generic.go:356] "Generic (PLEG): container finished" podID="bb4565b0-e59b-448c-ab3d-f83fd88e8a62" containerID="fe1ecb4f127ffab6dd1a0790ef346eb831f6bdf20f033f1713d1d28c82982eb9" exitCode=0 Jun 11 18:36:26.165212 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:26.164800 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4qrvl" event={"ID":"bb4565b0-e59b-448c-ab3d-f83fd88e8a62","Type":"ContainerDied","Data":"fe1ecb4f127ffab6dd1a0790ef346eb831f6bdf20f033f1713d1d28c82982eb9"} Jun 11 18:36:26.913189 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:26.913144 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:26.913338 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:26.913144 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:26.913338 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:26.913264 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:26.913338 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:26.913284 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:26.913437 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:26.913406 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:26.913518 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:26.913497 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:27.171366 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:27.171292 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 18:36:27.171845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:27.171622 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" event={"ID":"597fee71-168a-4df0-b3c0-6d65f17a570f","Type":"ContainerStarted","Data":"e6b5f7f5952c098457cd068d1625cc9e0e1ffac1e06295192e5c154862b97b63"} Jun 11 18:36:28.913789 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:28.913548 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:28.914484 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:28.913631 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:28.914484 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:28.913885 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:28.914484 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:28.913951 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:28.914484 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:28.913675 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:28.914484 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:28.914083 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:29.179168 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:29.179098 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 18:36:29.179473 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:29.179450 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" event={"ID":"597fee71-168a-4df0-b3c0-6d65f17a570f","Type":"ContainerStarted","Data":"b251f81cc621acb5406c862f19d849db8608c2cf7a59d04434434b011d34cc60"} Jun 11 18:36:29.179721 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:29.179697 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:29.179873 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:29.179854 2561 scope.go:117] "RemoveContainer" containerID="ffe360825040069b204c6cd6b02e32133897b2ee946a948de1cf576448b6082a" Jun 11 18:36:29.195860 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:29.195842 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:30.186787 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:30.186756 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 18:36:30.187270 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:30.187135 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" event={"ID":"597fee71-168a-4df0-b3c0-6d65f17a570f","Type":"ContainerStarted","Data":"e1dedd791b83e7ef4498cf5abc3ad7f570645bb9b50481fbe8f603f2c3783277"} Jun 11 18:36:30.187495 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:30.187474 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:30.187591 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:30.187566 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:30.206431 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:30.206392 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:36:30.217852 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:30.217792 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" podStartSLOduration=9.60999225 podStartE2EDuration="31.217770718s" totalImagesPullingTime="21.607778468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 18:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:30.217241012 +0000 UTC m=+31.785314513" watchObservedRunningTime="2026-06-11 18:36:30.217770718 +0000 UTC m=+31.785844198" Jun 11 18:36:30.489211 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:30.489119 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-58lbd"] Jun 11 18:36:30.489369 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:30.489305 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:30.489446 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:30.489421 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:30.492739 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:30.492697 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fjvnc"] Jun 11 18:36:30.492892 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:30.492860 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:30.493034 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:30.492987 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:30.493412 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:30.493391 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dk499"] Jun 11 18:36:30.493509 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:30.493496 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:30.493622 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:30.493599 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:31.912766 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:31.912723 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:31.912766 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:31.912751 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:31.913302 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:31.912751 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:31.913302 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:31.912864 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:31.913302 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:31.912996 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:31.913302 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:31.913099 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:32.195183 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:32.195087 2561 generic.go:356] "Generic (PLEG): container finished" podID="bb4565b0-e59b-448c-ab3d-f83fd88e8a62" containerID="e5cdfee71111de8d0aab2160d0363b5a112f0ab47465dd3a4b137a05bf43f7b3" exitCode=0 Jun 11 18:36:32.195183 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:32.195162 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4qrvl" event={"ID":"bb4565b0-e59b-448c-ab3d-f83fd88e8a62","Type":"ContainerDied","Data":"e5cdfee71111de8d0aab2160d0363b5a112f0ab47465dd3a4b137a05bf43f7b3"} Jun 11 18:36:32.605302 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:32.605265 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs\") pod \"network-metrics-daemon-fjvnc\" (UID: \"0a9668ae-c995-40a8-b7fd-de034a8fff15\") " pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:32.605499 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:32.605439 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 18:36:32.605576 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:32.605520 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs podName:0a9668ae-c995-40a8-b7fd-de034a8fff15 nodeName:}" failed. No retries permitted until 2026-06-11 18:37:04.605497529 +0000 UTC m=+66.173571002 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs") pod "network-metrics-daemon-fjvnc" (UID: "0a9668ae-c995-40a8-b7fd-de034a8fff15") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 18:36:32.706074 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:32.705991 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:32.706238 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:32.706126 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 18:36:32.706238 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:32.706210 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret podName:fb731d8a-734b-48f1-87a2-303368226673 nodeName:}" failed. No retries permitted until 2026-06-11 18:37:04.706168607 +0000 UTC m=+66.274242068 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret") pod "global-pull-secret-syncer-58lbd" (UID: "fb731d8a-734b-48f1-87a2-303368226673") : object "kube-system"/"original-pull-secret" not registered Jun 11 18:36:32.807383 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:32.807349 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frv7t\" (UniqueName: \"kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t\") pod \"network-check-target-dk499\" (UID: \"67b90c12-c8b0-4ceb-a8df-8369f7fff5a0\") " pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:32.807539 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:32.807509 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 18:36:32.807539 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:32.807531 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 18:36:32.807605 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:32.807541 2561 projected.go:196] Error preparing data for projected volume kube-api-access-frv7t for pod openshift-network-diagnostics/network-check-target-dk499: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 18:36:32.807605 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:32.807590 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t podName:67b90c12-c8b0-4ceb-a8df-8369f7fff5a0 nodeName:}" failed. No retries permitted until 2026-06-11 18:37:04.807576356 +0000 UTC m=+66.375649814 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-frv7t" (UniqueName: "kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t") pod "network-check-target-dk499" (UID: "67b90c12-c8b0-4ceb-a8df-8369f7fff5a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 18:36:33.199833 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:33.199593 2561 generic.go:356] "Generic (PLEG): container finished" podID="bb4565b0-e59b-448c-ab3d-f83fd88e8a62" containerID="4d9245f9cb63fe15568696c43e614a8315829754ac3c93961820ab0680cebeb1" exitCode=0 Jun 11 18:36:33.200215 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:33.199668 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4qrvl" event={"ID":"bb4565b0-e59b-448c-ab3d-f83fd88e8a62","Type":"ContainerDied","Data":"4d9245f9cb63fe15568696c43e614a8315829754ac3c93961820ab0680cebeb1"} Jun 11 18:36:33.912919 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:33.912888 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:33.913084 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:33.912936 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:33.913084 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:33.912898 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:33.913084 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:33.913007 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:33.913202 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:33.913088 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:33.913202 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:33.913169 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:34.204914 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:34.204833 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4qrvl" event={"ID":"bb4565b0-e59b-448c-ab3d-f83fd88e8a62","Type":"ContainerStarted","Data":"9b53b98f216aa584198a3e1a2cfb954c272169180f809fcc66fc7f28410cab0a"} Jun 11 18:36:34.227163 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:34.227082 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4qrvl" podStartSLOduration=7.474991359 podStartE2EDuration="35.227064622s" totalImagesPullingTime="27.752073263s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 18:35:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:34.225930921 +0000 UTC m=+35.794004412" watchObservedRunningTime="2026-06-11 18:36:34.227064622 +0000 UTC m=+35.795138105" Jun 11 18:36:35.912208 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:35.912157 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:35.912627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:35.912165 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:35.912627 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:35.912328 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjvnc" podUID="0a9668ae-c995-40a8-b7fd-de034a8fff15" Jun 11 18:36:35.912627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:35.912164 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:35.912627 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:35.912360 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dk499" podUID="67b90c12-c8b0-4ceb-a8df-8369f7fff5a0" Jun 11 18:36:35.912627 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:35.912438 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58lbd" podUID="fb731d8a-734b-48f1-87a2-303368226673" Jun 11 18:36:37.232838 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.232764 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-156.ec2.internal" event="NodeReady" Jun 11 18:36:37.233352 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.232888 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 11 18:36:37.272373 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.272341 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d7cdb9796-wnw7p"] Jun 11 18:36:37.275794 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.275772 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.278381 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.278352 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 11 18:36:37.278499 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.278362 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 11 18:36:37.278499 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.278391 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 11 18:36:37.278499 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.278409 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qs799\"" Jun 11 18:36:37.286328 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.286312 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 11 18:36:37.289712 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.289668 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pxnqh"] Jun 11 18:36:37.294087 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.294068 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-p9n54"] Jun 11 18:36:37.294160 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.294148 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:37.296322 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.296258 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nhg9x\"" Jun 11 18:36:37.296820 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.296807 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 11 18:36:37.297568 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.297551 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 11 18:36:37.297996 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.297981 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt"] Jun 11 18:36:37.298089 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.298076 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-p9n54" Jun 11 18:36:37.300696 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.300672 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 11 18:36:37.300796 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.300705 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-j5zsm\"" Jun 11 18:36:37.300796 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.300673 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 11 18:36:37.301593 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.301573 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s"] Jun 11 18:36:37.301692 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.301679 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:36:37.304864 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.304845 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2hjj6\"" Jun 11 18:36:37.305307 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.305288 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 11 18:36:37.305414 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.305391 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 11 18:36:37.305466 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.305396 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5dbbb5d744-kpstz"] Jun 11 18:36:37.305466 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.305459 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" Jun 11 18:36:37.307230 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.307207 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 11 18:36:37.312267 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.311798 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 11 18:36:37.312267 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.312002 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 11 18:36:37.312950 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.312515 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 11 18:36:37.312950 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.312798 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 11 18:36:37.313434 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.313414 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zlbgx\"" Jun 11 18:36:37.315692 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.315605 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg"] Jun 11 18:36:37.316792 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.316769 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:37.319611 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.319592 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 11 18:36:37.319725 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.319633 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 11 18:36:37.319960 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.319897 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-8xnrq\"" Jun 11 18:36:37.320255 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.320234 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 11 18:36:37.320406 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.320384 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d7cdb9796-wnw7p"] Jun 11 18:36:37.320528 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.320440 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pxnqh"] Jun 11 18:36:37.320528 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.320471 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-p9n54"] Jun 11 18:36:37.320528 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.320483 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 11 18:36:37.320528 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.320495 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5bbd86d6bd-h7sf5"] Jun 11 18:36:37.320792 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.320626 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" Jun 11 18:36:37.324364 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.324343 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg"] Jun 11 18:36:37.324475 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.324427 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.324475 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.324469 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt"] Jun 11 18:36:37.324584 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.324502 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh"] Jun 11 18:36:37.324989 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.324961 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jun 11 18:36:37.325091 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.325064 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-87z59\"" Jun 11 18:36:37.325147 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.325118 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jun 11 18:36:37.325355 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.325338 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jun 11 18:36:37.325642 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.325627 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 11 18:36:37.326346 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.326312 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jun 11 18:36:37.328656 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.328284 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-867b76cb74-szglk"] Jun 11 18:36:37.328656 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.328350 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lmwpc\"" Jun 11 18:36:37.328905 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.328745 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jun 11 18:36:37.329061 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.329034 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 11 18:36:37.329061 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.329051 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 11 18:36:37.329254 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.329034 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jun 11 18:36:37.329449 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.329430 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" Jun 11 18:36:37.332507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.332488 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 11 18:36:37.332868 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.332636 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 11 18:36:37.334448 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.332994 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6947749cfc-lzkjq"] Jun 11 18:36:37.334448 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.333389 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" Jun 11 18:36:37.334730 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.334708 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 11 18:36:37.334894 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.334874 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 11 18:36:37.334971 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.334874 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-mx7n7\"" Jun 11 18:36:37.336545 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.336472 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 11 18:36:37.336640 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.336616 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5dbbb5d744-kpstz"] Jun 11 18:36:37.336705 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.336644 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s"] Jun 11 18:36:37.336705 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.336672 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-85b5fb789b-jt8gg"] Jun 11 18:36:37.336911 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.336893 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6947749cfc-lzkjq" Jun 11 18:36:37.337073 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.337053 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 11 18:36:37.337487 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.337461 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-55k76\"" Jun 11 18:36:37.337734 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.337714 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jun 11 18:36:37.339561 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.339538 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 18:36:37.339561 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.339549 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 18:36:37.339686 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.339607 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-nk294\"" Jun 11 18:36:37.340342 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.340325 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tt7bw"] Jun 11 18:36:37.340467 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.340451 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:37.342757 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.342584 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 11 18:36:37.342757 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.342588 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 11 18:36:37.342757 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.342662 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 11 18:36:37.343008 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.342992 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 11 18:36:37.343105 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.343085 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 11 18:36:37.344139 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.343440 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-sk7rt\"" Jun 11 18:36:37.344139 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.343934 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 11 18:36:37.344555 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.344429 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5bbd86d6bd-h7sf5"] Jun 11 18:36:37.344555 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.344470 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tt7bw"] Jun 11 18:36:37.344555 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.344508 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-867b76cb74-szglk"] Jun 11 18:36:37.344555 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.344532 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6947749cfc-lzkjq"] Jun 11 18:36:37.344794 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.344559 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh"] Jun 11 18:36:37.344794 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.344584 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-85b5fb789b-jt8gg"] Jun 11 18:36:37.344794 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.344614 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tt7bw" Jun 11 18:36:37.346567 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.346544 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 11 18:36:37.346756 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.346738 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nsdln\"" Jun 11 18:36:37.347042 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.347025 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 11 18:36:37.347124 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.347056 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 11 18:36:37.445368 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445331 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4de0340-920b-472d-8ff1-0f6b69840a5e-serving-cert\") pod \"console-operator-5dbbb5d744-kpstz\" (UID: \"d4de0340-920b-472d-8ff1-0f6b69840a5e\") " pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:37.445554 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445378 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64cd27df-ed04-40a9-89c5-0df92402e31b-service-ca-bundle\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.445554 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445421 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp5fx\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-kube-api-access-mp5fx\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.445554 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445456 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:37.445554 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445474 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3745889d-fbde-41e9-b083-3a33484ded3a-nginx-conf\") pod \"networking-console-plugin-867b76cb74-szglk\" (UID: \"3745889d-fbde-41e9-b083-3a33484ded3a\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" Jun 11 18:36:37.445554 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445489 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f2b34d7-67c4-4695-b705-40f852c4f63f-cert\") pod \"ingress-canary-tt7bw\" (UID: \"6f2b34d7-67c4-4695-b705-40f852c4f63f\") " pod="openshift-ingress-canary/ingress-canary-tt7bw" Jun 11 18:36:37.445554 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445514 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2ldv\" (UniqueName: \"kubernetes.io/projected/c1651923-19d8-41b4-8f23-21afb8ac43a7-kube-api-access-h2ldv\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:37.445834 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445559 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6f7bd047-0864-4400-8fe8-dada6f2e942b-ca-trust-extracted\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.445834 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445587 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64cd27df-ed04-40a9-89c5-0df92402e31b-trusted-ca-bundle\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.445834 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445626 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-szglk\" (UID: \"3745889d-fbde-41e9-b083-3a33484ded3a\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" Jun 11 18:36:37.445834 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445654 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:37.445834 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445722 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:37.445834 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445750 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e42d52fc-e45a-415f-9694-26853b83c2be-tmp-dir\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:37.445834 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445776 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-certificates\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.445834 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445800 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6f7bd047-0864-4400-8fe8-dada6f2e942b-installation-pull-secrets\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.445834 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445836 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2lqr\" (UniqueName: \"kubernetes.io/projected/8e0dc434-3567-48d2-980c-1747934625aa-kube-api-access-h2lqr\") pod \"service-ca-operator-7ccbfd59dc-b96qg\" (UID: \"8e0dc434-3567-48d2-980c-1747934625aa\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" Jun 11 18:36:37.446345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445861 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgdwp\" (UniqueName: \"kubernetes.io/projected/64cd27df-ed04-40a9-89c5-0df92402e31b-kube-api-access-bgdwp\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.446345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445892 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-85bbt\" (UID: \"08e48b3b-9d88-4715-aa6a-008e8cdebb3e\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:36:37.446345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445933 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-bound-sa-token\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.446345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445960 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf7lp\" (UniqueName: \"kubernetes.io/projected/6f2b34d7-67c4-4695-b705-40f852c4f63f-kube-api-access-cf7lp\") pod \"ingress-canary-tt7bw\" (UID: \"6f2b34d7-67c4-4695-b705-40f852c4f63f\") " pod="openshift-ingress-canary/ingress-canary-tt7bw" Jun 11 18:36:37.446345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.445985 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-default-certificate\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:37.446345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446012 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znssw\" (UniqueName: \"kubernetes.io/projected/d4de0340-920b-472d-8ff1-0f6b69840a5e-kube-api-access-znssw\") pod \"console-operator-5dbbb5d744-kpstz\" (UID: \"d4de0340-920b-472d-8ff1-0f6b69840a5e\") " pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:37.446345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446039 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-dgg2s\" (UID: \"5c322acb-2426-4224-a49a-68ad890205bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" Jun 11 18:36:37.446345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446067 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwv69\" (UniqueName: \"kubernetes.io/projected/6df2530a-816a-4737-8815-4c537d8a61c4-kube-api-access-bwv69\") pod \"kube-storage-version-migrator-operator-7c578b78fc-6kgqh\" (UID: \"6df2530a-816a-4737-8815-4c537d8a61c4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" Jun 11 18:36:37.446345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446091 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6f7bd047-0864-4400-8fe8-dada6f2e942b-image-registry-private-configuration\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.446345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446121 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6df2530a-816a-4737-8815-4c537d8a61c4-serving-cert\") pod \"kube-storage-version-migrator-operator-7c578b78fc-6kgqh\" (UID: \"6df2530a-816a-4737-8815-4c537d8a61c4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" Jun 11 18:36:37.446345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446144 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.446345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446192 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-telemetry-config\") pod \"cluster-monitoring-operator-f9d7df769-85bbt\" (UID: \"08e48b3b-9d88-4715-aa6a-008e8cdebb3e\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:36:37.446345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446218 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/64cd27df-ed04-40a9-89c5-0df92402e31b-snapshots\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.446345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446244 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e42d52fc-e45a-415f-9694-26853b83c2be-config-volume\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:37.446345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446267 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e0dc434-3567-48d2-980c-1747934625aa-config\") pod \"service-ca-operator-7ccbfd59dc-b96qg\" (UID: \"8e0dc434-3567-48d2-980c-1747934625aa\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" Jun 11 18:36:37.446871 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446290 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5slx7\" (UniqueName: \"kubernetes.io/projected/fa8ba06f-6538-4c77-977c-2a07af6e04d8-kube-api-access-5slx7\") pod \"network-check-source-6947749cfc-lzkjq\" (UID: \"fa8ba06f-6538-4c77-977c-2a07af6e04d8\") " pod="openshift-network-diagnostics/network-check-source-6947749cfc-lzkjq" Jun 11 18:36:37.446871 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446313 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g2pw\" (UniqueName: \"kubernetes.io/projected/5c322acb-2426-4224-a49a-68ad890205bc-kube-api-access-9g2pw\") pod \"cluster-samples-operator-9547488fd-dgg2s\" (UID: \"5c322acb-2426-4224-a49a-68ad890205bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" Jun 11 18:36:37.446871 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446354 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f7bd047-0864-4400-8fe8-dada6f2e942b-trusted-ca\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.446871 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446390 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnwnx\" (UniqueName: \"kubernetes.io/projected/e42d52fc-e45a-415f-9694-26853b83c2be-kube-api-access-gnwnx\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:37.446871 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446417 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64cd27df-ed04-40a9-89c5-0df92402e31b-tmp\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.446871 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446442 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/64cd27df-ed04-40a9-89c5-0df92402e31b-serving-cert\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.446871 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446469 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6df2530a-816a-4737-8815-4c537d8a61c4-config\") pod \"kube-storage-version-migrator-operator-7c578b78fc-6kgqh\" (UID: \"6df2530a-816a-4737-8815-4c537d8a61c4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" Jun 11 18:36:37.446871 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446497 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dtw8\" (UniqueName: \"kubernetes.io/projected/51cdc3b6-a443-43aa-b6c2-def980560f1b-kube-api-access-6dtw8\") pod \"volume-data-source-validator-bd4ccd799-p9n54\" (UID: \"51cdc3b6-a443-43aa-b6c2-def980560f1b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-p9n54" Jun 11 18:36:37.446871 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446521 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4de0340-920b-472d-8ff1-0f6b69840a5e-config\") pod \"console-operator-5dbbb5d744-kpstz\" (UID: \"d4de0340-920b-472d-8ff1-0f6b69840a5e\") " pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:37.446871 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446547 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4de0340-920b-472d-8ff1-0f6b69840a5e-trusted-ca\") pod \"console-operator-5dbbb5d744-kpstz\" (UID: \"d4de0340-920b-472d-8ff1-0f6b69840a5e\") " pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:37.446871 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446572 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e0dc434-3567-48d2-980c-1747934625aa-serving-cert\") pod \"service-ca-operator-7ccbfd59dc-b96qg\" (UID: \"8e0dc434-3567-48d2-980c-1747934625aa\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" Jun 11 18:36:37.446871 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446592 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlwcb\" (UniqueName: \"kubernetes.io/projected/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-kube-api-access-dlwcb\") pod \"cluster-monitoring-operator-f9d7df769-85bbt\" (UID: \"08e48b3b-9d88-4715-aa6a-008e8cdebb3e\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:36:37.446871 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.446608 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-stats-auth\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:37.547959 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.547871 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f7bd047-0864-4400-8fe8-dada6f2e942b-trusted-ca\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.547959 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.547905 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnwnx\" (UniqueName: \"kubernetes.io/projected/e42d52fc-e45a-415f-9694-26853b83c2be-kube-api-access-gnwnx\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:37.547959 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.547925 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64cd27df-ed04-40a9-89c5-0df92402e31b-tmp\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.548273 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548039 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/64cd27df-ed04-40a9-89c5-0df92402e31b-serving-cert\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.548273 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548071 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6df2530a-816a-4737-8815-4c537d8a61c4-config\") pod \"kube-storage-version-migrator-operator-7c578b78fc-6kgqh\" (UID: \"6df2530a-816a-4737-8815-4c537d8a61c4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" Jun 11 18:36:37.548273 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548098 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dtw8\" (UniqueName: \"kubernetes.io/projected/51cdc3b6-a443-43aa-b6c2-def980560f1b-kube-api-access-6dtw8\") pod \"volume-data-source-validator-bd4ccd799-p9n54\" (UID: \"51cdc3b6-a443-43aa-b6c2-def980560f1b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-p9n54" Jun 11 18:36:37.548273 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548123 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4de0340-920b-472d-8ff1-0f6b69840a5e-config\") pod \"console-operator-5dbbb5d744-kpstz\" (UID: \"d4de0340-920b-472d-8ff1-0f6b69840a5e\") " pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:37.548273 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548198 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4de0340-920b-472d-8ff1-0f6b69840a5e-trusted-ca\") pod \"console-operator-5dbbb5d744-kpstz\" (UID: \"d4de0340-920b-472d-8ff1-0f6b69840a5e\") " pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:37.548273 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548241 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e0dc434-3567-48d2-980c-1747934625aa-serving-cert\") pod \"service-ca-operator-7ccbfd59dc-b96qg\" (UID: \"8e0dc434-3567-48d2-980c-1747934625aa\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" Jun 11 18:36:37.548557 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548484 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64cd27df-ed04-40a9-89c5-0df92402e31b-tmp\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.548557 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548537 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlwcb\" (UniqueName: \"kubernetes.io/projected/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-kube-api-access-dlwcb\") pod \"cluster-monitoring-operator-f9d7df769-85bbt\" (UID: \"08e48b3b-9d88-4715-aa6a-008e8cdebb3e\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:36:37.548664 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548565 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-stats-auth\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:37.548664 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548636 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4de0340-920b-472d-8ff1-0f6b69840a5e-serving-cert\") pod \"console-operator-5dbbb5d744-kpstz\" (UID: \"d4de0340-920b-472d-8ff1-0f6b69840a5e\") " pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:37.548758 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548709 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64cd27df-ed04-40a9-89c5-0df92402e31b-service-ca-bundle\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.548812 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548771 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp5fx\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-kube-api-access-mp5fx\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.548812 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548802 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:37.548904 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548829 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6df2530a-816a-4737-8815-4c537d8a61c4-config\") pod \"kube-storage-version-migrator-operator-7c578b78fc-6kgqh\" (UID: \"6df2530a-816a-4737-8815-4c537d8a61c4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" Jun 11 18:36:37.548904 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548842 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3745889d-fbde-41e9-b083-3a33484ded3a-nginx-conf\") pod \"networking-console-plugin-867b76cb74-szglk\" (UID: \"3745889d-fbde-41e9-b083-3a33484ded3a\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" Jun 11 18:36:37.548904 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548849 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4de0340-920b-472d-8ff1-0f6b69840a5e-config\") pod \"console-operator-5dbbb5d744-kpstz\" (UID: \"d4de0340-920b-472d-8ff1-0f6b69840a5e\") " pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:37.548904 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548879 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f2b34d7-67c4-4695-b705-40f852c4f63f-cert\") pod \"ingress-canary-tt7bw\" (UID: \"6f2b34d7-67c4-4695-b705-40f852c4f63f\") " pod="openshift-ingress-canary/ingress-canary-tt7bw" Jun 11 18:36:37.549089 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548907 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2ldv\" (UniqueName: \"kubernetes.io/projected/c1651923-19d8-41b4-8f23-21afb8ac43a7-kube-api-access-h2ldv\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:37.549089 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548931 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f7bd047-0864-4400-8fe8-dada6f2e942b-trusted-ca\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.549089 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548938 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6f7bd047-0864-4400-8fe8-dada6f2e942b-ca-trust-extracted\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.549089 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548965 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64cd27df-ed04-40a9-89c5-0df92402e31b-trusted-ca-bundle\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.549089 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.548989 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-szglk\" (UID: \"3745889d-fbde-41e9-b083-3a33484ded3a\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" Jun 11 18:36:37.549089 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549014 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:37.549089 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549054 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:37.549089 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549084 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e42d52fc-e45a-415f-9694-26853b83c2be-tmp-dir\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:37.549408 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549107 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-certificates\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.549408 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549133 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6f7bd047-0864-4400-8fe8-dada6f2e942b-installation-pull-secrets\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.549408 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549194 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2lqr\" (UniqueName: \"kubernetes.io/projected/8e0dc434-3567-48d2-980c-1747934625aa-kube-api-access-h2lqr\") pod \"service-ca-operator-7ccbfd59dc-b96qg\" (UID: \"8e0dc434-3567-48d2-980c-1747934625aa\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" Jun 11 18:36:37.549408 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549220 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgdwp\" (UniqueName: \"kubernetes.io/projected/64cd27df-ed04-40a9-89c5-0df92402e31b-kube-api-access-bgdwp\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.549408 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549252 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-85bbt\" (UID: \"08e48b3b-9d88-4715-aa6a-008e8cdebb3e\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:36:37.549408 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:37.549276 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 18:36:37.549408 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549299 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-bound-sa-token\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.549408 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:37.549333 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls podName:e42d52fc-e45a-415f-9694-26853b83c2be nodeName:}" failed. No retries permitted until 2026-06-11 18:36:38.049315272 +0000 UTC m=+39.617388753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls") pod "dns-default-pxnqh" (UID: "e42d52fc-e45a-415f-9694-26853b83c2be") : secret "dns-default-metrics-tls" not found Jun 11 18:36:37.549408 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549352 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf7lp\" (UniqueName: \"kubernetes.io/projected/6f2b34d7-67c4-4695-b705-40f852c4f63f-kube-api-access-cf7lp\") pod \"ingress-canary-tt7bw\" (UID: \"6f2b34d7-67c4-4695-b705-40f852c4f63f\") " pod="openshift-ingress-canary/ingress-canary-tt7bw" Jun 11 18:36:37.549408 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549369 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64cd27df-ed04-40a9-89c5-0df92402e31b-service-ca-bundle\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.549408 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549380 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-default-certificate\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:37.549845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549415 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znssw\" (UniqueName: \"kubernetes.io/projected/d4de0340-920b-472d-8ff1-0f6b69840a5e-kube-api-access-znssw\") pod \"console-operator-5dbbb5d744-kpstz\" (UID: \"d4de0340-920b-472d-8ff1-0f6b69840a5e\") " pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:37.549845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549440 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-dgg2s\" (UID: \"5c322acb-2426-4224-a49a-68ad890205bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" Jun 11 18:36:37.549845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549470 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwv69\" (UniqueName: \"kubernetes.io/projected/6df2530a-816a-4737-8815-4c537d8a61c4-kube-api-access-bwv69\") pod \"kube-storage-version-migrator-operator-7c578b78fc-6kgqh\" (UID: \"6df2530a-816a-4737-8815-4c537d8a61c4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" Jun 11 18:36:37.549845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549550 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6f7bd047-0864-4400-8fe8-dada6f2e942b-image-registry-private-configuration\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.549845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549583 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6df2530a-816a-4737-8815-4c537d8a61c4-serving-cert\") pod \"kube-storage-version-migrator-operator-7c578b78fc-6kgqh\" (UID: \"6df2530a-816a-4737-8815-4c537d8a61c4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" Jun 11 18:36:37.549845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549609 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.549845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549639 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-telemetry-config\") pod \"cluster-monitoring-operator-f9d7df769-85bbt\" (UID: \"08e48b3b-9d88-4715-aa6a-008e8cdebb3e\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:36:37.549845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549663 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/64cd27df-ed04-40a9-89c5-0df92402e31b-snapshots\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.549845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549694 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e42d52fc-e45a-415f-9694-26853b83c2be-config-volume\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:37.549845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549720 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e0dc434-3567-48d2-980c-1747934625aa-config\") pod \"service-ca-operator-7ccbfd59dc-b96qg\" (UID: \"8e0dc434-3567-48d2-980c-1747934625aa\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" Jun 11 18:36:37.549845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549747 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5slx7\" (UniqueName: \"kubernetes.io/projected/fa8ba06f-6538-4c77-977c-2a07af6e04d8-kube-api-access-5slx7\") pod \"network-check-source-6947749cfc-lzkjq\" (UID: \"fa8ba06f-6538-4c77-977c-2a07af6e04d8\") " pod="openshift-network-diagnostics/network-check-source-6947749cfc-lzkjq" Jun 11 18:36:37.549845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549774 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g2pw\" (UniqueName: \"kubernetes.io/projected/5c322acb-2426-4224-a49a-68ad890205bc-kube-api-access-9g2pw\") pod \"cluster-samples-operator-9547488fd-dgg2s\" (UID: \"5c322acb-2426-4224-a49a-68ad890205bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" Jun 11 18:36:37.550328 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.549854 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3745889d-fbde-41e9-b083-3a33484ded3a-nginx-conf\") pod \"networking-console-plugin-867b76cb74-szglk\" (UID: \"3745889d-fbde-41e9-b083-3a33484ded3a\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" Jun 11 18:36:37.550328 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:37.550217 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 11 18:36:37.550328 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:37.550271 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls podName:5c322acb-2426-4224-a49a-68ad890205bc nodeName:}" failed. No retries permitted until 2026-06-11 18:36:38.050257959 +0000 UTC m=+39.618331432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls") pod "cluster-samples-operator-9547488fd-dgg2s" (UID: "5c322acb-2426-4224-a49a-68ad890205bc") : secret "samples-operator-tls" not found Jun 11 18:36:37.550454 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.550425 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6f7bd047-0864-4400-8fe8-dada6f2e942b-ca-trust-extracted\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.550871 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:37.550758 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 18:36:37.550871 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:37.550819 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert podName:3745889d-fbde-41e9-b083-3a33484ded3a nodeName:}" failed. No retries permitted until 2026-06-11 18:36:38.050802806 +0000 UTC m=+39.618876274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-szglk" (UID: "3745889d-fbde-41e9-b083-3a33484ded3a") : secret "networking-console-plugin-cert" not found Jun 11 18:36:37.551257 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.551231 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-certificates\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.551373 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:37.551358 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle podName:c1651923-19d8-41b4-8f23-21afb8ac43a7 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:38.051343543 +0000 UTC m=+39.619417025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle") pod "router-default-85b5fb789b-jt8gg" (UID: "c1651923-19d8-41b4-8f23-21afb8ac43a7") : configmap references non-existent config key: service-ca.crt Jun 11 18:36:37.551470 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.551444 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64cd27df-ed04-40a9-89c5-0df92402e31b-trusted-ca-bundle\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.551736 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.551712 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4de0340-920b-472d-8ff1-0f6b69840a5e-trusted-ca\") pod \"console-operator-5dbbb5d744-kpstz\" (UID: \"d4de0340-920b-472d-8ff1-0f6b69840a5e\") " pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:37.551820 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:37.551806 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 18:36:37.551878 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:37.551849 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls podName:08e48b3b-9d88-4715-aa6a-008e8cdebb3e nodeName:}" failed. No retries permitted until 2026-06-11 18:36:38.051834416 +0000 UTC m=+39.619907889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-85bbt" (UID: "08e48b3b-9d88-4715-aa6a-008e8cdebb3e") : secret "cluster-monitoring-operator-tls" not found Jun 11 18:36:37.552035 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:37.552017 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 18:36:37.552119 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:37.552038 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7cdb9796-wnw7p: secret "image-registry-tls" not found Jun 11 18:36:37.552119 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:37.552077 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls podName:6f7bd047-0864-4400-8fe8-dada6f2e942b nodeName:}" failed. No retries permitted until 2026-06-11 18:36:38.052063923 +0000 UTC m=+39.620137387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls") pod "image-registry-d7cdb9796-wnw7p" (UID: "6f7bd047-0864-4400-8fe8-dada6f2e942b") : secret "image-registry-tls" not found Jun 11 18:36:37.552520 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.552475 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/64cd27df-ed04-40a9-89c5-0df92402e31b-snapshots\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.552605 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.552494 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e0dc434-3567-48d2-980c-1747934625aa-config\") pod \"service-ca-operator-7ccbfd59dc-b96qg\" (UID: \"8e0dc434-3567-48d2-980c-1747934625aa\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" Jun 11 18:36:37.552730 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:37.552699 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 18:36:37.552795 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:37.552773 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs podName:c1651923-19d8-41b4-8f23-21afb8ac43a7 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:38.052758452 +0000 UTC m=+39.620831928 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs") pod "router-default-85b5fb789b-jt8gg" (UID: "c1651923-19d8-41b4-8f23-21afb8ac43a7") : secret "router-metrics-certs-default" not found Jun 11 18:36:37.552795 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.552773 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-telemetry-config\") pod \"cluster-monitoring-operator-f9d7df769-85bbt\" (UID: \"08e48b3b-9d88-4715-aa6a-008e8cdebb3e\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:36:37.553047 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.553000 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e42d52fc-e45a-415f-9694-26853b83c2be-tmp-dir\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:37.553226 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.553205 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e42d52fc-e45a-415f-9694-26853b83c2be-config-volume\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:37.553800 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.553775 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6f7bd047-0864-4400-8fe8-dada6f2e942b-image-registry-private-configuration\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.553800 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.553792 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f2b34d7-67c4-4695-b705-40f852c4f63f-cert\") pod \"ingress-canary-tt7bw\" (UID: \"6f2b34d7-67c4-4695-b705-40f852c4f63f\") " pod="openshift-ingress-canary/ingress-canary-tt7bw" Jun 11 18:36:37.554422 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.554372 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e0dc434-3567-48d2-980c-1747934625aa-serving-cert\") pod \"service-ca-operator-7ccbfd59dc-b96qg\" (UID: \"8e0dc434-3567-48d2-980c-1747934625aa\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" Jun 11 18:36:37.554574 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.554556 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-stats-auth\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:37.554717 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.554698 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-default-certificate\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:37.554850 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.554834 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6df2530a-816a-4737-8815-4c537d8a61c4-serving-cert\") pod \"kube-storage-version-migrator-operator-7c578b78fc-6kgqh\" (UID: \"6df2530a-816a-4737-8815-4c537d8a61c4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" Jun 11 18:36:37.555146 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.555126 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4de0340-920b-472d-8ff1-0f6b69840a5e-serving-cert\") pod \"console-operator-5dbbb5d744-kpstz\" (UID: \"d4de0340-920b-472d-8ff1-0f6b69840a5e\") " pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:37.555921 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.555898 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/64cd27df-ed04-40a9-89c5-0df92402e31b-serving-cert\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.555994 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.555920 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6f7bd047-0864-4400-8fe8-dada6f2e942b-installation-pull-secrets\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.557650 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.557626 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp5fx\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-kube-api-access-mp5fx\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.557911 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.557894 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dtw8\" (UniqueName: \"kubernetes.io/projected/51cdc3b6-a443-43aa-b6c2-def980560f1b-kube-api-access-6dtw8\") pod \"volume-data-source-validator-bd4ccd799-p9n54\" (UID: \"51cdc3b6-a443-43aa-b6c2-def980560f1b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-p9n54" Jun 11 18:36:37.563253 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.563228 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnwnx\" (UniqueName: \"kubernetes.io/projected/e42d52fc-e45a-415f-9694-26853b83c2be-kube-api-access-gnwnx\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:37.565071 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.565046 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-bound-sa-token\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:37.565294 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.565273 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwv69\" (UniqueName: \"kubernetes.io/projected/6df2530a-816a-4737-8815-4c537d8a61c4-kube-api-access-bwv69\") pod \"kube-storage-version-migrator-operator-7c578b78fc-6kgqh\" (UID: \"6df2530a-816a-4737-8815-4c537d8a61c4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" Jun 11 18:36:37.565372 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.565357 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgdwp\" (UniqueName: \"kubernetes.io/projected/64cd27df-ed04-40a9-89c5-0df92402e31b-kube-api-access-bgdwp\") pod \"insights-operator-5bbd86d6bd-h7sf5\" (UID: \"64cd27df-ed04-40a9-89c5-0df92402e31b\") " pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.568223 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.568202 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2lqr\" (UniqueName: \"kubernetes.io/projected/8e0dc434-3567-48d2-980c-1747934625aa-kube-api-access-h2lqr\") pod \"service-ca-operator-7ccbfd59dc-b96qg\" (UID: \"8e0dc434-3567-48d2-980c-1747934625aa\") " pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" Jun 11 18:36:37.572429 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.572405 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5slx7\" (UniqueName: \"kubernetes.io/projected/fa8ba06f-6538-4c77-977c-2a07af6e04d8-kube-api-access-5slx7\") pod \"network-check-source-6947749cfc-lzkjq\" (UID: \"fa8ba06f-6538-4c77-977c-2a07af6e04d8\") " pod="openshift-network-diagnostics/network-check-source-6947749cfc-lzkjq" Jun 11 18:36:37.574033 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.574011 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g2pw\" (UniqueName: \"kubernetes.io/projected/5c322acb-2426-4224-a49a-68ad890205bc-kube-api-access-9g2pw\") pod \"cluster-samples-operator-9547488fd-dgg2s\" (UID: \"5c322acb-2426-4224-a49a-68ad890205bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" Jun 11 18:36:37.575302 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.575282 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znssw\" (UniqueName: \"kubernetes.io/projected/d4de0340-920b-472d-8ff1-0f6b69840a5e-kube-api-access-znssw\") pod \"console-operator-5dbbb5d744-kpstz\" (UID: \"d4de0340-920b-472d-8ff1-0f6b69840a5e\") " pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:37.575838 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.575813 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlwcb\" (UniqueName: \"kubernetes.io/projected/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-kube-api-access-dlwcb\") pod \"cluster-monitoring-operator-f9d7df769-85bbt\" (UID: \"08e48b3b-9d88-4715-aa6a-008e8cdebb3e\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:36:37.575975 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.575950 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2ldv\" (UniqueName: \"kubernetes.io/projected/c1651923-19d8-41b4-8f23-21afb8ac43a7-kube-api-access-h2ldv\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:37.576186 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.576154 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf7lp\" (UniqueName: \"kubernetes.io/projected/6f2b34d7-67c4-4695-b705-40f852c4f63f-kube-api-access-cf7lp\") pod \"ingress-canary-tt7bw\" (UID: \"6f2b34d7-67c4-4695-b705-40f852c4f63f\") " pod="openshift-ingress-canary/ingress-canary-tt7bw" Jun 11 18:36:37.615725 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.615705 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-p9n54" Jun 11 18:36:37.621342 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.621308 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8zpf_21a1280a-bea2-4fe2-8795-a55d4dc7c65b/konnectivity-agent/0.log" limit=1024 Jun 11 18:36:37.647206 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.645343 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:37.653249 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.653142 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" Jun 11 18:36:37.657607 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.657485 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" Jun 11 18:36:37.662430 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.662404 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" Jun 11 18:36:37.674750 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.674707 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6947749cfc-lzkjq" Jun 11 18:36:37.684954 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.684918 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tt7bw" Jun 11 18:36:37.707378 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.707116 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8zpf_21a1280a-bea2-4fe2-8795-a55d4dc7c65b/konnectivity-agent/0.log" limit=1024 Jun 11 18:36:37.855665 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.855570 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-p9n54"] Jun 11 18:36:37.861931 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.861891 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5dbbb5d744-kpstz"] Jun 11 18:36:37.863832 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:37.863805 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51cdc3b6_a443_43aa_b6c2_def980560f1b.slice/crio-ac955ec36d1b68827a39f492c07ae9fcda74d1f28677f5d68931d838d37836f7 WatchSource:0}: Error finding container ac955ec36d1b68827a39f492c07ae9fcda74d1f28677f5d68931d838d37836f7: Status 404 returned error can't find the container with id ac955ec36d1b68827a39f492c07ae9fcda74d1f28677f5d68931d838d37836f7 Jun 11 18:36:37.866340 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:37.866225 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4de0340_920b_472d_8ff1_0f6b69840a5e.slice/crio-6176f6c957533eac2f98c82dffcce74918c63a7229a7709737dfade9fcbfd750 WatchSource:0}: Error finding container 6176f6c957533eac2f98c82dffcce74918c63a7229a7709737dfade9fcbfd750: Status 404 returned error can't find the container with id 6176f6c957533eac2f98c82dffcce74918c63a7229a7709737dfade9fcbfd750 Jun 11 18:36:37.875528 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.871939 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg"] Jun 11 18:36:37.875528 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:37.874507 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e0dc434_3567_48d2_980c_1747934625aa.slice/crio-a47f1a52a739a78131bb5c6a2f2761c4eb7087fcacab845a2f4191876ef8fb1d WatchSource:0}: Error finding container a47f1a52a739a78131bb5c6a2f2761c4eb7087fcacab845a2f4191876ef8fb1d: Status 404 returned error can't find the container with id a47f1a52a739a78131bb5c6a2f2761c4eb7087fcacab845a2f4191876ef8fb1d Jun 11 18:36:37.897186 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.897145 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6947749cfc-lzkjq"] Jun 11 18:36:37.900313 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:37.900284 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa8ba06f_6538_4c77_977c_2a07af6e04d8.slice/crio-67cbda06df9761b5faf51556054f19029d1134e5554c25ed4f0cc697af192de4 WatchSource:0}: Error finding container 67cbda06df9761b5faf51556054f19029d1134e5554c25ed4f0cc697af192de4: Status 404 returned error can't find the container with id 67cbda06df9761b5faf51556054f19029d1134e5554c25ed4f0cc697af192de4 Jun 11 18:36:37.900591 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.900570 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh"] Jun 11 18:36:37.903800 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:37.903774 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df2530a_816a_4737_8815_4c537d8a61c4.slice/crio-f178a4dd6112a498cd90c6673a4266b3b1b2a13ee4a8f48b517b15ae0952856a WatchSource:0}: Error finding container f178a4dd6112a498cd90c6673a4266b3b1b2a13ee4a8f48b517b15ae0952856a: Status 404 returned error can't find the container with id f178a4dd6112a498cd90c6673a4266b3b1b2a13ee4a8f48b517b15ae0952856a Jun 11 18:36:37.906206 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.906081 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tt7bw"] Jun 11 18:36:37.910166 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:37.910056 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f2b34d7_67c4_4695_b705_40f852c4f63f.slice/crio-e8537b30626f21524f56bc5d6390c5dda4966cf72f853ac81c7c947d36dbf513 WatchSource:0}: Error finding container e8537b30626f21524f56bc5d6390c5dda4966cf72f853ac81c7c947d36dbf513: Status 404 returned error can't find the container with id e8537b30626f21524f56bc5d6390c5dda4966cf72f853ac81c7c947d36dbf513 Jun 11 18:36:37.912093 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.912073 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:36:37.912207 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.912112 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:36:37.912207 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.912198 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:36:37.914695 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.914677 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qf6vw\"" Jun 11 18:36:37.914827 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.914695 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 18:36:37.914827 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.914709 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ptdz8\"" Jun 11 18:36:37.914827 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.914745 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 18:36:37.914827 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:37.914699 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5bbd86d6bd-h7sf5"] Jun 11 18:36:37.918151 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:37.918132 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64cd27df_ed04_40a9_89c5_0df92402e31b.slice/crio-4355b6a13c64f7e510b72ac4131498d9ff9a35ad005ec56d36ea0914169e2fa4 WatchSource:0}: Error finding container 4355b6a13c64f7e510b72ac4131498d9ff9a35ad005ec56d36ea0914169e2fa4: Status 404 returned error can't find the container with id 4355b6a13c64f7e510b72ac4131498d9ff9a35ad005ec56d36ea0914169e2fa4 Jun 11 18:36:38.054599 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:38.054570 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:38.054808 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:38.054623 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-85bbt\" (UID: \"08e48b3b-9d88-4715-aa6a-008e8cdebb3e\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:36:38.054808 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:38.054681 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-dgg2s\" (UID: \"5c322acb-2426-4224-a49a-68ad890205bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" Jun 11 18:36:38.054808 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:38.054711 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:38.054808 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:38.054729 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 18:36:38.054808 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:38.054791 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 18:36:38.054808 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:38.054800 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs podName:c1651923-19d8-41b4-8f23-21afb8ac43a7 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:39.054781563 +0000 UTC m=+40.622855035 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs") pod "router-default-85b5fb789b-jt8gg" (UID: "c1651923-19d8-41b4-8f23-21afb8ac43a7") : secret "router-metrics-certs-default" not found Jun 11 18:36:38.055115 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:38.054791 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 11 18:36:38.055115 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:38.054836 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls podName:08e48b3b-9d88-4715-aa6a-008e8cdebb3e nodeName:}" failed. No retries permitted until 2026-06-11 18:36:39.054824731 +0000 UTC m=+40.622898188 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-85bbt" (UID: "08e48b3b-9d88-4715-aa6a-008e8cdebb3e") : secret "cluster-monitoring-operator-tls" not found Jun 11 18:36:38.055115 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:38.054840 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 18:36:38.055115 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:38.054855 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7cdb9796-wnw7p: secret "image-registry-tls" not found Jun 11 18:36:38.055115 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:38.054856 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:38.055115 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:38.054866 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls podName:5c322acb-2426-4224-a49a-68ad890205bc nodeName:}" failed. No retries permitted until 2026-06-11 18:36:39.054855308 +0000 UTC m=+40.622928783 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls") pod "cluster-samples-operator-9547488fd-dgg2s" (UID: "5c322acb-2426-4224-a49a-68ad890205bc") : secret "samples-operator-tls" not found Jun 11 18:36:38.055115 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:38.054887 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls podName:6f7bd047-0864-4400-8fe8-dada6f2e942b nodeName:}" failed. No retries permitted until 2026-06-11 18:36:39.054879087 +0000 UTC m=+40.622952550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls") pod "image-registry-d7cdb9796-wnw7p" (UID: "6f7bd047-0864-4400-8fe8-dada6f2e942b") : secret "image-registry-tls" not found Jun 11 18:36:38.055115 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:38.054903 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 18:36:38.055115 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:38.054920 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-szglk\" (UID: \"3745889d-fbde-41e9-b083-3a33484ded3a\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" Jun 11 18:36:38.055115 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:38.054927 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls podName:e42d52fc-e45a-415f-9694-26853b83c2be nodeName:}" failed. No retries permitted until 2026-06-11 18:36:39.054920468 +0000 UTC m=+40.622993926 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls") pod "dns-default-pxnqh" (UID: "e42d52fc-e45a-415f-9694-26853b83c2be") : secret "dns-default-metrics-tls" not found Jun 11 18:36:38.055115 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:38.054955 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:38.055115 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:38.054970 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 18:36:38.055115 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:38.054992 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert podName:3745889d-fbde-41e9-b083-3a33484ded3a nodeName:}" failed. No retries permitted until 2026-06-11 18:36:39.054985501 +0000 UTC m=+40.623058959 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-szglk" (UID: "3745889d-fbde-41e9-b083-3a33484ded3a") : secret "networking-console-plugin-cert" not found Jun 11 18:36:38.055115 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:38.055088 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle podName:c1651923-19d8-41b4-8f23-21afb8ac43a7 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:39.055079079 +0000 UTC m=+40.623152540 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle") pod "router-default-85b5fb789b-jt8gg" (UID: "c1651923-19d8-41b4-8f23-21afb8ac43a7") : configmap references non-existent config key: service-ca.crt Jun 11 18:36:38.216287 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:38.216202 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6947749cfc-lzkjq" event={"ID":"fa8ba06f-6538-4c77-977c-2a07af6e04d8","Type":"ContainerStarted","Data":"67cbda06df9761b5faf51556054f19029d1134e5554c25ed4f0cc697af192de4"} Jun 11 18:36:38.217282 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:38.217250 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-p9n54" event={"ID":"51cdc3b6-a443-43aa-b6c2-def980560f1b","Type":"ContainerStarted","Data":"ac955ec36d1b68827a39f492c07ae9fcda74d1f28677f5d68931d838d37836f7"} Jun 11 18:36:38.218425 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:38.218405 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tt7bw" event={"ID":"6f2b34d7-67c4-4695-b705-40f852c4f63f","Type":"ContainerStarted","Data":"e8537b30626f21524f56bc5d6390c5dda4966cf72f853ac81c7c947d36dbf513"} Jun 11 18:36:38.219494 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:38.219471 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" event={"ID":"8e0dc434-3567-48d2-980c-1747934625aa","Type":"ContainerStarted","Data":"a47f1a52a739a78131bb5c6a2f2761c4eb7087fcacab845a2f4191876ef8fb1d"} Jun 11 18:36:38.220466 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:38.220442 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" event={"ID":"6df2530a-816a-4737-8815-4c537d8a61c4","Type":"ContainerStarted","Data":"f178a4dd6112a498cd90c6673a4266b3b1b2a13ee4a8f48b517b15ae0952856a"} Jun 11 18:36:38.221468 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:38.221449 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" event={"ID":"64cd27df-ed04-40a9-89c5-0df92402e31b","Type":"ContainerStarted","Data":"4355b6a13c64f7e510b72ac4131498d9ff9a35ad005ec56d36ea0914169e2fa4"} Jun 11 18:36:38.222473 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:38.222455 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" event={"ID":"d4de0340-920b-472d-8ff1-0f6b69840a5e","Type":"ContainerStarted","Data":"6176f6c957533eac2f98c82dffcce74918c63a7229a7709737dfade9fcbfd750"} Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:39.070348 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:39.070397 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-szglk\" (UID: \"3745889d-fbde-41e9-b083-3a33484ded3a\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:39.070429 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:39.070465 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:39.070504 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-85bbt\" (UID: \"08e48b3b-9d88-4715-aa6a-008e8cdebb3e\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:39.070548 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-dgg2s\" (UID: \"5c322acb-2426-4224-a49a-68ad890205bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:39.070579 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:39.070749 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:39.070762 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7cdb9796-wnw7p: secret "image-registry-tls" not found Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:39.070815 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls podName:6f7bd047-0864-4400-8fe8-dada6f2e942b nodeName:}" failed. No retries permitted until 2026-06-11 18:36:41.070798668 +0000 UTC m=+42.638872131 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls") pod "image-registry-d7cdb9796-wnw7p" (UID: "6f7bd047-0864-4400-8fe8-dada6f2e942b") : secret "image-registry-tls" not found Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:39.071225 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:39.071273 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls podName:e42d52fc-e45a-415f-9694-26853b83c2be nodeName:}" failed. No retries permitted until 2026-06-11 18:36:41.07125763 +0000 UTC m=+42.639331094 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls") pod "dns-default-pxnqh" (UID: "e42d52fc-e45a-415f-9694-26853b83c2be") : secret "dns-default-metrics-tls" not found Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:39.071335 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:39.071367 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert podName:3745889d-fbde-41e9-b083-3a33484ded3a nodeName:}" failed. No retries permitted until 2026-06-11 18:36:41.071355837 +0000 UTC m=+42.639429299 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-szglk" (UID: "3745889d-fbde-41e9-b083-3a33484ded3a") : secret "networking-console-plugin-cert" not found Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:39.071436 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle podName:c1651923-19d8-41b4-8f23-21afb8ac43a7 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:41.071425874 +0000 UTC m=+42.639499338 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle") pod "router-default-85b5fb789b-jt8gg" (UID: "c1651923-19d8-41b4-8f23-21afb8ac43a7") : configmap references non-existent config key: service-ca.crt Jun 11 18:36:39.071731 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:39.071490 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 18:36:39.073031 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:39.071519 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs podName:c1651923-19d8-41b4-8f23-21afb8ac43a7 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:41.071509225 +0000 UTC m=+42.639582685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs") pod "router-default-85b5fb789b-jt8gg" (UID: "c1651923-19d8-41b4-8f23-21afb8ac43a7") : secret "router-metrics-certs-default" not found Jun 11 18:36:39.073031 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:39.071573 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 18:36:39.073031 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:39.071603 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls podName:08e48b3b-9d88-4715-aa6a-008e8cdebb3e nodeName:}" failed. No retries permitted until 2026-06-11 18:36:41.071594515 +0000 UTC m=+42.639667981 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-85bbt" (UID: "08e48b3b-9d88-4715-aa6a-008e8cdebb3e") : secret "cluster-monitoring-operator-tls" not found Jun 11 18:36:39.073031 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:39.071655 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 11 18:36:39.073031 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:39.071686 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls podName:5c322acb-2426-4224-a49a-68ad890205bc nodeName:}" failed. No retries permitted until 2026-06-11 18:36:41.071676248 +0000 UTC m=+42.639749706 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls") pod "cluster-samples-operator-9547488fd-dgg2s" (UID: "5c322acb-2426-4224-a49a-68ad890205bc") : secret "samples-operator-tls" not found Jun 11 18:36:39.684491 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:39.684438 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8zpf_21a1280a-bea2-4fe2-8795-a55d4dc7c65b/konnectivity-agent/0.log" limit=1024 Jun 11 18:36:41.092126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:41.092076 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-dgg2s\" (UID: \"5c322acb-2426-4224-a49a-68ad890205bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" Jun 11 18:36:41.092594 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:41.092244 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:41.092594 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:41.092260 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 11 18:36:41.092594 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:41.092368 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:41.092594 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:41.092405 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-szglk\" (UID: \"3745889d-fbde-41e9-b083-3a33484ded3a\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" Jun 11 18:36:41.092594 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:41.092430 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls podName:5c322acb-2426-4224-a49a-68ad890205bc nodeName:}" failed. No retries permitted until 2026-06-11 18:36:45.092404599 +0000 UTC m=+46.660478063 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls") pod "cluster-samples-operator-9547488fd-dgg2s" (UID: "5c322acb-2426-4224-a49a-68ad890205bc") : secret "samples-operator-tls" not found Jun 11 18:36:41.092594 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:41.092460 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 18:36:41.092594 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:41.092473 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:41.092594 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:41.092507 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert podName:3745889d-fbde-41e9-b083-3a33484ded3a nodeName:}" failed. No retries permitted until 2026-06-11 18:36:45.092495669 +0000 UTC m=+46.660569127 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-szglk" (UID: "3745889d-fbde-41e9-b083-3a33484ded3a") : secret "networking-console-plugin-cert" not found Jun 11 18:36:41.092594 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:41.092539 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:41.092594 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:41.092545 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 18:36:41.092594 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:41.092567 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle podName:c1651923-19d8-41b4-8f23-21afb8ac43a7 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:45.092555303 +0000 UTC m=+46.660628764 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle") pod "router-default-85b5fb789b-jt8gg" (UID: "c1651923-19d8-41b4-8f23-21afb8ac43a7") : configmap references non-existent config key: service-ca.crt Jun 11 18:36:41.092594 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:41.092319 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 18:36:41.092594 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:41.092594 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 18:36:41.093202 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:41.092594 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7cdb9796-wnw7p: secret "image-registry-tls" not found Jun 11 18:36:41.093202 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:41.092596 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls podName:e42d52fc-e45a-415f-9694-26853b83c2be nodeName:}" failed. No retries permitted until 2026-06-11 18:36:45.092581635 +0000 UTC m=+46.660655094 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls") pod "dns-default-pxnqh" (UID: "e42d52fc-e45a-415f-9694-26853b83c2be") : secret "dns-default-metrics-tls" not found Jun 11 18:36:41.093202 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:41.092703 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs podName:c1651923-19d8-41b4-8f23-21afb8ac43a7 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:45.092691772 +0000 UTC m=+46.660765231 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs") pod "router-default-85b5fb789b-jt8gg" (UID: "c1651923-19d8-41b4-8f23-21afb8ac43a7") : secret "router-metrics-certs-default" not found Jun 11 18:36:41.093202 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:41.092723 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls podName:6f7bd047-0864-4400-8fe8-dada6f2e942b nodeName:}" failed. No retries permitted until 2026-06-11 18:36:45.092712844 +0000 UTC m=+46.660786308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls") pod "image-registry-d7cdb9796-wnw7p" (UID: "6f7bd047-0864-4400-8fe8-dada6f2e942b") : secret "image-registry-tls" not found Jun 11 18:36:41.093202 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:41.092757 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-85bbt\" (UID: \"08e48b3b-9d88-4715-aa6a-008e8cdebb3e\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:36:41.093202 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:41.092851 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 18:36:41.093202 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:41.092887 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls podName:08e48b3b-9d88-4715-aa6a-008e8cdebb3e nodeName:}" failed. No retries permitted until 2026-06-11 18:36:45.092876623 +0000 UTC m=+46.660950084 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-85bbt" (UID: "08e48b3b-9d88-4715-aa6a-008e8cdebb3e") : secret "cluster-monitoring-operator-tls" not found Jun 11 18:36:45.128260 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:45.128223 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-85bbt\" (UID: \"08e48b3b-9d88-4715-aa6a-008e8cdebb3e\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:36:45.128679 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:45.128300 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-dgg2s\" (UID: \"5c322acb-2426-4224-a49a-68ad890205bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" Jun 11 18:36:45.128679 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:45.128340 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:45.128679 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:45.128356 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 18:36:45.128679 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:45.128436 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls podName:08e48b3b-9d88-4715-aa6a-008e8cdebb3e nodeName:}" failed. No retries permitted until 2026-06-11 18:36:53.128414863 +0000 UTC m=+54.696488344 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-85bbt" (UID: "08e48b3b-9d88-4715-aa6a-008e8cdebb3e") : secret "cluster-monitoring-operator-tls" not found Jun 11 18:36:45.128679 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:45.128650 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:45.128945 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:45.128689 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-szglk\" (UID: \"3745889d-fbde-41e9-b083-3a33484ded3a\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" Jun 11 18:36:45.128945 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:45.128717 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:45.128945 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:45.128767 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:45.128945 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:45.128847 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 18:36:45.128945 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:45.128899 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls podName:e42d52fc-e45a-415f-9694-26853b83c2be nodeName:}" failed. No retries permitted until 2026-06-11 18:36:53.128883923 +0000 UTC m=+54.696957399 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls") pod "dns-default-pxnqh" (UID: "e42d52fc-e45a-415f-9694-26853b83c2be") : secret "dns-default-metrics-tls" not found Jun 11 18:36:45.128945 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:45.128902 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 18:36:45.129269 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:45.128951 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs podName:c1651923-19d8-41b4-8f23-21afb8ac43a7 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:53.128936545 +0000 UTC m=+54.697010010 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs") pod "router-default-85b5fb789b-jt8gg" (UID: "c1651923-19d8-41b4-8f23-21afb8ac43a7") : secret "router-metrics-certs-default" not found Jun 11 18:36:45.129269 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:45.128951 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 11 18:36:45.129269 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:45.128970 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 18:36:45.129269 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:45.128983 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7cdb9796-wnw7p: secret "image-registry-tls" not found Jun 11 18:36:45.129269 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:45.129004 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls podName:5c322acb-2426-4224-a49a-68ad890205bc nodeName:}" failed. No retries permitted until 2026-06-11 18:36:53.128991473 +0000 UTC m=+54.697064954 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls") pod "cluster-samples-operator-9547488fd-dgg2s" (UID: "5c322acb-2426-4224-a49a-68ad890205bc") : secret "samples-operator-tls" not found Jun 11 18:36:45.129269 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:45.129014 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 18:36:45.129269 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:45.129023 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls podName:6f7bd047-0864-4400-8fe8-dada6f2e942b nodeName:}" failed. No retries permitted until 2026-06-11 18:36:53.129014577 +0000 UTC m=+54.697088034 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls") pod "image-registry-d7cdb9796-wnw7p" (UID: "6f7bd047-0864-4400-8fe8-dada6f2e942b") : secret "image-registry-tls" not found Jun 11 18:36:45.129269 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:45.129049 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert podName:3745889d-fbde-41e9-b083-3a33484ded3a nodeName:}" failed. No retries permitted until 2026-06-11 18:36:53.129036362 +0000 UTC m=+54.697109838 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-szglk" (UID: "3745889d-fbde-41e9-b083-3a33484ded3a") : secret "networking-console-plugin-cert" not found Jun 11 18:36:45.129269 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:45.129076 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle podName:c1651923-19d8-41b4-8f23-21afb8ac43a7 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:53.129064209 +0000 UTC m=+54.697137685 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle") pod "router-default-85b5fb789b-jt8gg" (UID: "c1651923-19d8-41b4-8f23-21afb8ac43a7") : configmap references non-existent config key: service-ca.crt Jun 11 18:36:45.341925 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:45.341816 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8zpf_21a1280a-bea2-4fe2-8795-a55d4dc7c65b/konnectivity-agent/0.log" limit=1024 Jun 11 18:36:46.032638 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.032591 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8zpf_21a1280a-bea2-4fe2-8795-a55d4dc7c65b/konnectivity-agent/0.log" limit=1024 Jun 11 18:36:46.131258 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.131221 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8zpf_21a1280a-bea2-4fe2-8795-a55d4dc7c65b/konnectivity-agent/0.log" limit=1024 Jun 11 18:36:46.250910 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.250876 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" event={"ID":"64cd27df-ed04-40a9-89c5-0df92402e31b","Type":"ContainerStarted","Data":"26acb70c5d7fd7eb0470fe6ef3936c06ee7d1962f0dd3a46c31f2ef13c3f6bf5"} Jun 11 18:36:46.253092 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.253070 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/0.log" Jun 11 18:36:46.253247 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.253103 2561 generic.go:356] "Generic (PLEG): container finished" podID="d4de0340-920b-472d-8ff1-0f6b69840a5e" containerID="8d2712cc5706acc8734d1bc15b8c581603e8e94bb1b33c087df82d7813a8b058" exitCode=255 Jun 11 18:36:46.253349 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.253327 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" event={"ID":"d4de0340-920b-472d-8ff1-0f6b69840a5e","Type":"ContainerDied","Data":"8d2712cc5706acc8734d1bc15b8c581603e8e94bb1b33c087df82d7813a8b058"} Jun 11 18:36:46.253437 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.253426 2561 scope.go:117] "RemoveContainer" containerID="8d2712cc5706acc8734d1bc15b8c581603e8e94bb1b33c087df82d7813a8b058" Jun 11 18:36:46.255265 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.255242 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6947749cfc-lzkjq" event={"ID":"fa8ba06f-6538-4c77-977c-2a07af6e04d8","Type":"ContainerStarted","Data":"8a8b9165029af2b1ce32f3b51ed1ac3e4bbc0723296131e4e82f52e3f38c0a51"} Jun 11 18:36:46.257297 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.257048 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-p9n54" event={"ID":"51cdc3b6-a443-43aa-b6c2-def980560f1b","Type":"ContainerStarted","Data":"4c9295f18a53d6b2d19c491282025b9183c3ba39e21233ce1626261a777289fd"} Jun 11 18:36:46.258680 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.258652 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tt7bw" event={"ID":"6f2b34d7-67c4-4695-b705-40f852c4f63f","Type":"ContainerStarted","Data":"1e58b27693bfea99b424a9d5ae5a4e411be8eb1153811cd9d3a479dceed92014"} Jun 11 18:36:46.260211 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.260145 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" event={"ID":"8e0dc434-3567-48d2-980c-1747934625aa","Type":"ContainerStarted","Data":"896f749ca3f15fc58b992078abaf1854edb31af223676eb4efe03c518f4030a9"} Jun 11 18:36:46.261808 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.261783 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" event={"ID":"6df2530a-816a-4737-8815-4c537d8a61c4","Type":"ContainerStarted","Data":"42377da449cad637e6c69a42bcdb6050b8cde2ef4e63519e29dc8927724a231b"} Jun 11 18:36:46.272912 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.272857 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" podStartSLOduration=10.063485149 podStartE2EDuration="17.27283975s" totalImagesPullingTime="7.209354601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:46.271015197 +0000 UTC m=+47.839088689" watchObservedRunningTime="2026-06-11 18:36:46.27283975 +0000 UTC m=+47.840913223" Jun 11 18:36:46.290323 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.290102 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-bd4ccd799-p9n54" podStartSLOduration=11.082427561 podStartE2EDuration="18.29007937s" totalImagesPullingTime="7.207651809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:36:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:46.288215863 +0000 UTC m=+47.856289370" watchObservedRunningTime="2026-06-11 18:36:46.29007937 +0000 UTC m=+47.858152850" Jun 11 18:36:46.328725 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.328670 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" podStartSLOduration=10.107401003 podStartE2EDuration="17.328648138s" totalImagesPullingTime="7.221247135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:46.327684011 +0000 UTC m=+47.895757491" watchObservedRunningTime="2026-06-11 18:36:46.328648138 +0000 UTC m=+47.896721622" Jun 11 18:36:46.348050 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.348005 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tt7bw" podStartSLOduration=2.12918184 podStartE2EDuration="9.347985883s" totalImagesPullingTime="7.218804043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:46.347504183 +0000 UTC m=+47.915577665" watchObservedRunningTime="2026-06-11 18:36:46.347985883 +0000 UTC m=+47.916059368" Jun 11 18:36:46.388544 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.388486 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" podStartSLOduration=10.135963205 podStartE2EDuration="17.388463687s" totalImagesPullingTime="7.252500482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:46.367280713 +0000 UTC m=+47.935354218" watchObservedRunningTime="2026-06-11 18:36:46.388463687 +0000 UTC m=+47.956537169" Jun 11 18:36:46.388933 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.388905 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6947749cfc-lzkjq" podStartSLOduration=10.151447561 podStartE2EDuration="17.388891468s" totalImagesPullingTime="7.237443907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:46.387746092 +0000 UTC m=+47.955819573" watchObservedRunningTime="2026-06-11 18:36:46.388891468 +0000 UTC m=+47.956964949" Jun 11 18:36:46.412210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.412163 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8zpf_21a1280a-bea2-4fe2-8795-a55d4dc7c65b/konnectivity-agent/0.log" limit=1024 Jun 11 18:36:46.512374 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:46.512334 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8zpf_21a1280a-bea2-4fe2-8795-a55d4dc7c65b/konnectivity-agent/0.log" limit=1024 Jun 11 18:36:47.161191 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.161141 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8zpf_21a1280a-bea2-4fe2-8795-a55d4dc7c65b/konnectivity-agent/0.log" limit=1024 Jun 11 18:36:47.226589 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.226556 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d4bcd4867-6pz4b"] Jun 11 18:36:47.242929 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.242903 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-6pz4b" Jun 11 18:36:47.245396 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.245376 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 11 18:36:47.246153 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.246133 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-p7xt7\"" Jun 11 18:36:47.246272 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.246168 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 11 18:36:47.247972 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.247949 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d4bcd4867-6pz4b"] Jun 11 18:36:47.271814 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.271788 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 18:36:47.272389 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.272364 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/0.log" Jun 11 18:36:47.272519 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.272413 2561 generic.go:356] "Generic (PLEG): container finished" podID="d4de0340-920b-472d-8ff1-0f6b69840a5e" containerID="df0de100bf003c615472461f1450ded54e444f71f41744466dfed9c4a71675a3" exitCode=255 Jun 11 18:36:47.272583 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.272529 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" event={"ID":"d4de0340-920b-472d-8ff1-0f6b69840a5e","Type":"ContainerDied","Data":"df0de100bf003c615472461f1450ded54e444f71f41744466dfed9c4a71675a3"} Jun 11 18:36:47.272634 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.272607 2561 scope.go:117] "RemoveContainer" containerID="8d2712cc5706acc8734d1bc15b8c581603e8e94bb1b33c087df82d7813a8b058" Jun 11 18:36:47.273134 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.273112 2561 scope.go:117] "RemoveContainer" containerID="df0de100bf003c615472461f1450ded54e444f71f41744466dfed9c4a71675a3" Jun 11 18:36:47.273390 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:47.273362 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5dbbb5d744-kpstz_openshift-console-operator(d4de0340-920b-472d-8ff1-0f6b69840a5e)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" podUID="d4de0340-920b-472d-8ff1-0f6b69840a5e" Jun 11 18:36:47.287141 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.287109 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8zpf_21a1280a-bea2-4fe2-8795-a55d4dc7c65b/konnectivity-agent/0.log" limit=1024 Jun 11 18:36:47.307396 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.307364 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jjtxg"] Jun 11 18:36:47.329958 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.329931 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jjtxg"] Jun 11 18:36:47.330094 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.330053 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.332159 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.332134 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 11 18:36:47.332159 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.332148 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 11 18:36:47.332362 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.332341 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rbv24\"" Jun 11 18:36:47.350814 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.350791 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.350965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.350857 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f504b49-7bd7-4d7e-8de4-bcc282566088-data-volume\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.351487 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.351457 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0f504b49-7bd7-4d7e-8de4-bcc282566088-crio-socket\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.351606 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.351574 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.351749 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.351729 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6rnm\" (UniqueName: \"kubernetes.io/projected/b3f6a6e3-662f-4b78-b930-f87e6e78c8c6-kube-api-access-q6rnm\") pod \"migrator-d4bcd4867-6pz4b\" (UID: \"b3f6a6e3-662f-4b78-b930-f87e6e78c8c6\") " pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-6pz4b" Jun 11 18:36:47.351900 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.351881 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htxkx\" (UniqueName: \"kubernetes.io/projected/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-api-access-htxkx\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.360397 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.360367 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8zpf_21a1280a-bea2-4fe2-8795-a55d4dc7c65b/konnectivity-agent/0.log" limit=1024 Jun 11 18:36:47.452679 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.452586 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.452679 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.452629 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f504b49-7bd7-4d7e-8de4-bcc282566088-data-volume\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.452910 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:47.452713 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 18:36:47.452910 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.452778 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0f504b49-7bd7-4d7e-8de4-bcc282566088-crio-socket\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.452910 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:47.452793 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm podName:0f504b49-7bd7-4d7e-8de4-bcc282566088 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:47.952775385 +0000 UTC m=+49.520848843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jjtxg" (UID: "0f504b49-7bd7-4d7e-8de4-bcc282566088") : configmap "kube-rbac-proxy" not found Jun 11 18:36:47.452910 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.452839 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0f504b49-7bd7-4d7e-8de4-bcc282566088-crio-socket\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.452910 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.452851 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.453136 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.452931 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6rnm\" (UniqueName: \"kubernetes.io/projected/b3f6a6e3-662f-4b78-b930-f87e6e78c8c6-kube-api-access-q6rnm\") pod \"migrator-d4bcd4867-6pz4b\" (UID: \"b3f6a6e3-662f-4b78-b930-f87e6e78c8c6\") " pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-6pz4b" Jun 11 18:36:47.453136 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.452989 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f504b49-7bd7-4d7e-8de4-bcc282566088-data-volume\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.453136 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:47.453003 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 18:36:47.453136 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.453013 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htxkx\" (UniqueName: \"kubernetes.io/projected/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-api-access-htxkx\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.453136 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:47.453065 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls podName:0f504b49-7bd7-4d7e-8de4-bcc282566088 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:47.953053177 +0000 UTC m=+49.521126636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jjtxg" (UID: "0f504b49-7bd7-4d7e-8de4-bcc282566088") : secret "insights-runtime-extractor-tls" not found Jun 11 18:36:47.462588 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.462560 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6rnm\" (UniqueName: \"kubernetes.io/projected/b3f6a6e3-662f-4b78-b930-f87e6e78c8c6-kube-api-access-q6rnm\") pod \"migrator-d4bcd4867-6pz4b\" (UID: \"b3f6a6e3-662f-4b78-b930-f87e6e78c8c6\") " pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-6pz4b" Jun 11 18:36:47.462588 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.462578 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htxkx\" (UniqueName: \"kubernetes.io/projected/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-api-access-htxkx\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.465967 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.465939 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8zpf_21a1280a-bea2-4fe2-8795-a55d4dc7c65b/konnectivity-agent/0.log" limit=1024 Jun 11 18:36:47.531597 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.531529 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8zpf_21a1280a-bea2-4fe2-8795-a55d4dc7c65b/konnectivity-agent/0.log" limit=1024 Jun 11 18:36:47.553707 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.553672 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-6pz4b" Jun 11 18:36:47.646743 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.646712 2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:47.646897 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.646776 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:36:47.694462 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.694356 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d4bcd4867-6pz4b"] Jun 11 18:36:47.697106 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:47.697080 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3f6a6e3_662f_4b78_b930_f87e6e78c8c6.slice/crio-e35714ea0df9b30155a05b3da77ca17c07b0c92ea54f964a36b8de9fec1ae269 WatchSource:0}: Error finding container e35714ea0df9b30155a05b3da77ca17c07b0c92ea54f964a36b8de9fec1ae269: Status 404 returned error can't find the container with id e35714ea0df9b30155a05b3da77ca17c07b0c92ea54f964a36b8de9fec1ae269 Jun 11 18:36:47.956826 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.956793 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.957007 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:47.956916 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:47.957007 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:47.956960 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 18:36:47.957079 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:47.957029 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls podName:0f504b49-7bd7-4d7e-8de4-bcc282566088 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:48.957015135 +0000 UTC m=+50.525088593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jjtxg" (UID: "0f504b49-7bd7-4d7e-8de4-bcc282566088") : secret "insights-runtime-extractor-tls" not found Jun 11 18:36:47.957079 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:47.957035 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 18:36:47.957146 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:47.957084 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm podName:0f504b49-7bd7-4d7e-8de4-bcc282566088 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:48.957068992 +0000 UTC m=+50.525142471 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jjtxg" (UID: "0f504b49-7bd7-4d7e-8de4-bcc282566088") : configmap "kube-rbac-proxy" not found Jun 11 18:36:48.279263 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:48.279191 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 18:36:48.279691 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:48.279572 2561 scope.go:117] "RemoveContainer" containerID="df0de100bf003c615472461f1450ded54e444f71f41744466dfed9c4a71675a3" Jun 11 18:36:48.279793 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:48.279765 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5dbbb5d744-kpstz_openshift-console-operator(d4de0340-920b-472d-8ff1-0f6b69840a5e)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" podUID="d4de0340-920b-472d-8ff1-0f6b69840a5e" Jun 11 18:36:48.280650 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:48.280623 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-6pz4b" event={"ID":"b3f6a6e3-662f-4b78-b930-f87e6e78c8c6","Type":"ContainerStarted","Data":"e35714ea0df9b30155a05b3da77ca17c07b0c92ea54f964a36b8de9fec1ae269"} Jun 11 18:36:48.967937 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:48.967903 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:48.968095 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:48.968010 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:48.968095 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:48.968023 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 18:36:48.968095 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:48.968084 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm podName:0f504b49-7bd7-4d7e-8de4-bcc282566088 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:50.96806728 +0000 UTC m=+52.536140761 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jjtxg" (UID: "0f504b49-7bd7-4d7e-8de4-bcc282566088") : configmap "kube-rbac-proxy" not found Jun 11 18:36:48.968231 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:48.968134 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 18:36:48.968231 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:48.968201 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls podName:0f504b49-7bd7-4d7e-8de4-bcc282566088 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:50.968168555 +0000 UTC m=+52.536242016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jjtxg" (UID: "0f504b49-7bd7-4d7e-8de4-bcc282566088") : secret "insights-runtime-extractor-tls" not found Jun 11 18:36:49.285576 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.285545 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-6pz4b" event={"ID":"b3f6a6e3-662f-4b78-b930-f87e6e78c8c6","Type":"ContainerStarted","Data":"2bed0bbb4d13ade68fb8f37970415569df70d69887058ff0fc07658aab025719"} Jun 11 18:36:49.286016 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.285853 2561 scope.go:117] "RemoveContainer" containerID="df0de100bf003c615472461f1450ded54e444f71f41744466dfed9c4a71675a3" Jun 11 18:36:49.286079 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:49.286040 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5dbbb5d744-kpstz_openshift-console-operator(d4de0340-920b-472d-8ff1-0f6b69840a5e)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" podUID="d4de0340-920b-472d-8ff1-0f6b69840a5e" Jun 11 18:36:49.506900 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.506826 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w4hqm_3bac4fe8-0f72-4e10-8c74-3df40f3e6a05/dns-node-resolver/0.log" Jun 11 18:36:49.537479 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.537441 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-l49r8"] Jun 11 18:36:49.564185 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.564149 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-l49r8"] Jun 11 18:36:49.564345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.564272 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7b994bc59-l49r8" Jun 11 18:36:49.566801 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.566774 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 11 18:36:49.566899 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.566771 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-pp295\"" Jun 11 18:36:49.567471 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.567443 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 11 18:36:49.567471 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.567443 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 11 18:36:49.567623 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.567449 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 11 18:36:49.675148 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.675110 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67gxn\" (UniqueName: \"kubernetes.io/projected/8ede9626-06f3-43aa-89e0-02d2b9195448-kube-api-access-67gxn\") pod \"service-ca-7b994bc59-l49r8\" (UID: \"8ede9626-06f3-43aa-89e0-02d2b9195448\") " pod="openshift-service-ca/service-ca-7b994bc59-l49r8" Jun 11 18:36:49.675331 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.675313 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8ede9626-06f3-43aa-89e0-02d2b9195448-signing-cabundle\") pod \"service-ca-7b994bc59-l49r8\" (UID: \"8ede9626-06f3-43aa-89e0-02d2b9195448\") " pod="openshift-service-ca/service-ca-7b994bc59-l49r8" Jun 11 18:36:49.675397 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.675379 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8ede9626-06f3-43aa-89e0-02d2b9195448-signing-key\") pod \"service-ca-7b994bc59-l49r8\" (UID: \"8ede9626-06f3-43aa-89e0-02d2b9195448\") " pod="openshift-service-ca/service-ca-7b994bc59-l49r8" Jun 11 18:36:49.776506 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.776419 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8ede9626-06f3-43aa-89e0-02d2b9195448-signing-cabundle\") pod \"service-ca-7b994bc59-l49r8\" (UID: \"8ede9626-06f3-43aa-89e0-02d2b9195448\") " pod="openshift-service-ca/service-ca-7b994bc59-l49r8" Jun 11 18:36:49.776506 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.776482 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8ede9626-06f3-43aa-89e0-02d2b9195448-signing-key\") pod \"service-ca-7b994bc59-l49r8\" (UID: \"8ede9626-06f3-43aa-89e0-02d2b9195448\") " pod="openshift-service-ca/service-ca-7b994bc59-l49r8" Jun 11 18:36:49.776682 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.776545 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67gxn\" (UniqueName: \"kubernetes.io/projected/8ede9626-06f3-43aa-89e0-02d2b9195448-kube-api-access-67gxn\") pod \"service-ca-7b994bc59-l49r8\" (UID: \"8ede9626-06f3-43aa-89e0-02d2b9195448\") " pod="openshift-service-ca/service-ca-7b994bc59-l49r8" Jun 11 18:36:49.777119 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.777100 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8ede9626-06f3-43aa-89e0-02d2b9195448-signing-cabundle\") pod \"service-ca-7b994bc59-l49r8\" (UID: \"8ede9626-06f3-43aa-89e0-02d2b9195448\") " pod="openshift-service-ca/service-ca-7b994bc59-l49r8" Jun 11 18:36:49.778868 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.778850 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8ede9626-06f3-43aa-89e0-02d2b9195448-signing-key\") pod \"service-ca-7b994bc59-l49r8\" (UID: \"8ede9626-06f3-43aa-89e0-02d2b9195448\") " pod="openshift-service-ca/service-ca-7b994bc59-l49r8" Jun 11 18:36:49.784671 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.784638 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67gxn\" (UniqueName: \"kubernetes.io/projected/8ede9626-06f3-43aa-89e0-02d2b9195448-kube-api-access-67gxn\") pod \"service-ca-7b994bc59-l49r8\" (UID: \"8ede9626-06f3-43aa-89e0-02d2b9195448\") " pod="openshift-service-ca/service-ca-7b994bc59-l49r8" Jun 11 18:36:49.873856 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:49.873820 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7b994bc59-l49r8" Jun 11 18:36:50.029110 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:50.029032 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-l49r8"] Jun 11 18:36:50.031765 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:50.031741 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ede9626_06f3_43aa_89e0_02d2b9195448.slice/crio-9dfa88b585e72276d9e7f94efd4ecc66b8d03472355c5dc4222bc9e5fdc034ef WatchSource:0}: Error finding container 9dfa88b585e72276d9e7f94efd4ecc66b8d03472355c5dc4222bc9e5fdc034ef: Status 404 returned error can't find the container with id 9dfa88b585e72276d9e7f94efd4ecc66b8d03472355c5dc4222bc9e5fdc034ef Jun 11 18:36:50.106799 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:50.106779 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jx59l_ed57ef82-e664-4ffd-b72c-3fc0422eacab/node-ca/0.log" Jun 11 18:36:50.290004 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:50.289913 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7b994bc59-l49r8" event={"ID":"8ede9626-06f3-43aa-89e0-02d2b9195448","Type":"ContainerStarted","Data":"aa047fa74d9dd1982f5305132e4d138dd4741f25fd47447e22cb1b6aca223487"} Jun 11 18:36:50.290004 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:50.289966 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7b994bc59-l49r8" event={"ID":"8ede9626-06f3-43aa-89e0-02d2b9195448","Type":"ContainerStarted","Data":"9dfa88b585e72276d9e7f94efd4ecc66b8d03472355c5dc4222bc9e5fdc034ef"} Jun 11 18:36:50.291703 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:50.291680 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-6pz4b" event={"ID":"b3f6a6e3-662f-4b78-b930-f87e6e78c8c6","Type":"ContainerStarted","Data":"689c73334db0dba0647bf2e997437e02b7cf29bed7866870643144e2a831bdf0"} Jun 11 18:36:50.306873 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:50.306828 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7b994bc59-l49r8" podStartSLOduration=1.306812197 podStartE2EDuration="1.306812197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:36:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:50.306124983 +0000 UTC m=+51.874198484" watchObservedRunningTime="2026-06-11 18:36:50.306812197 +0000 UTC m=+51.874885711" Jun 11 18:36:50.324432 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:50.324376 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-6pz4b" podStartSLOduration=1.85437987 podStartE2EDuration="3.324361318s" totalImagesPullingTime="1.469981448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:50.323048027 +0000 UTC m=+51.891121504" watchObservedRunningTime="2026-06-11 18:36:50.324361318 +0000 UTC m=+51.892434846" Jun 11 18:36:50.906551 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:50.906524 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tt7bw_6f2b34d7-67c4-4695-b705-40f852c4f63f/serve-healthcheck-canary/0.log" Jun 11 18:36:50.988744 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:50.988715 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:50.988918 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:50.988853 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:50.988918 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:50.988867 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 18:36:50.988984 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:50.988932 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls podName:0f504b49-7bd7-4d7e-8de4-bcc282566088 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:54.988914284 +0000 UTC m=+56.556987756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jjtxg" (UID: "0f504b49-7bd7-4d7e-8de4-bcc282566088") : secret "insights-runtime-extractor-tls" not found Jun 11 18:36:50.988984 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:50.988932 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 18:36:50.988984 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:50.988965 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm podName:0f504b49-7bd7-4d7e-8de4-bcc282566088 nodeName:}" failed. No retries permitted until 2026-06-11 18:36:54.988957262 +0000 UTC m=+56.557030733 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jjtxg" (UID: "0f504b49-7bd7-4d7e-8de4-bcc282566088") : configmap "kube-rbac-proxy" not found Jun 11 18:36:51.107224 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:51.107193 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d4bcd4867-6pz4b_b3f6a6e3-662f-4b78-b930-f87e6e78c8c6/migrator/0.log" Jun 11 18:36:51.308009 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:51.307932 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d4bcd4867-6pz4b_b3f6a6e3-662f-4b78-b930-f87e6e78c8c6/graceful-termination/0.log" Jun 11 18:36:51.516890 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:51.516851 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7c578b78fc-6kgqh_6df2530a-816a-4737-8815-4c537d8a61c4/kube-storage-version-migrator-operator/0.log" Jun 11 18:36:53.212831 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:53.212795 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-szglk\" (UID: \"3745889d-fbde-41e9-b083-3a33484ded3a\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" Jun 11 18:36:53.213339 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:53.212846 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:53.213339 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:53.212884 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:36:53.213339 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:53.212922 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-85bbt\" (UID: \"08e48b3b-9d88-4715-aa6a-008e8cdebb3e\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:36:53.213339 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:53.212973 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-dgg2s\" (UID: \"5c322acb-2426-4224-a49a-68ad890205bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" Jun 11 18:36:53.213339 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:53.212978 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 18:36:53.213339 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:53.213011 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls\") pod \"image-registry-d7cdb9796-wnw7p\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:53.213339 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:53.213048 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 18:36:53.213339 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:53.213052 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 18:36:53.213339 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:53.213079 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert podName:3745889d-fbde-41e9-b083-3a33484ded3a nodeName:}" failed. No retries permitted until 2026-06-11 18:37:09.21305509 +0000 UTC m=+70.781128565 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert") pod "networking-console-plugin-867b76cb74-szglk" (UID: "3745889d-fbde-41e9-b083-3a33484ded3a") : secret "networking-console-plugin-cert" not found Jun 11 18:36:53.213339 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:53.213111 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls podName:08e48b3b-9d88-4715-aa6a-008e8cdebb3e nodeName:}" failed. No retries permitted until 2026-06-11 18:37:09.213096227 +0000 UTC m=+70.781169713 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-f9d7df769-85bbt" (UID: "08e48b3b-9d88-4715-aa6a-008e8cdebb3e") : secret "cluster-monitoring-operator-tls" not found Jun 11 18:36:53.213339 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:53.213135 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs podName:c1651923-19d8-41b4-8f23-21afb8ac43a7 nodeName:}" failed. No retries permitted until 2026-06-11 18:37:09.213126108 +0000 UTC m=+70.781199593 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs") pod "router-default-85b5fb789b-jt8gg" (UID: "c1651923-19d8-41b4-8f23-21afb8ac43a7") : secret "router-metrics-certs-default" not found Jun 11 18:36:53.213339 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:53.213144 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 18:36:53.213339 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:53.213151 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle podName:c1651923-19d8-41b4-8f23-21afb8ac43a7 nodeName:}" failed. No retries permitted until 2026-06-11 18:37:09.213142954 +0000 UTC m=+70.781216425 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle") pod "router-default-85b5fb789b-jt8gg" (UID: "c1651923-19d8-41b4-8f23-21afb8ac43a7") : configmap references non-existent config key: service-ca.crt Jun 11 18:36:53.213339 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:53.213159 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7cdb9796-wnw7p: secret "image-registry-tls" not found Jun 11 18:36:53.213339 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:53.213250 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls podName:6f7bd047-0864-4400-8fe8-dada6f2e942b nodeName:}" failed. No retries permitted until 2026-06-11 18:37:09.213234646 +0000 UTC m=+70.781308123 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls") pod "image-registry-d7cdb9796-wnw7p" (UID: "6f7bd047-0864-4400-8fe8-dada6f2e942b") : secret "image-registry-tls" not found Jun 11 18:36:53.213857 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:53.213271 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:36:53.213857 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:53.213378 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 18:36:53.213857 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:53.213421 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls podName:e42d52fc-e45a-415f-9694-26853b83c2be nodeName:}" failed. No retries permitted until 2026-06-11 18:37:09.213410015 +0000 UTC m=+70.781483487 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls") pod "dns-default-pxnqh" (UID: "e42d52fc-e45a-415f-9694-26853b83c2be") : secret "dns-default-metrics-tls" not found Jun 11 18:36:53.215422 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:53.215395 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c322acb-2426-4224-a49a-68ad890205bc-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-dgg2s\" (UID: \"5c322acb-2426-4224-a49a-68ad890205bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" Jun 11 18:36:53.234694 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:53.234675 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" Jun 11 18:36:53.431826 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:53.431793 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s"] Jun 11 18:36:54.307564 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:54.307526 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" event={"ID":"5c322acb-2426-4224-a49a-68ad890205bc","Type":"ContainerStarted","Data":"0fb987809b342275b5bcce6c1653e19a1ed6152b81a1e63abe28b07ac896e414"} Jun 11 18:36:55.031507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:55.031467 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:55.031683 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:55.031616 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:36:55.031683 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:55.031633 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 18:36:55.031801 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:55.031743 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls podName:0f504b49-7bd7-4d7e-8de4-bcc282566088 nodeName:}" failed. No retries permitted until 2026-06-11 18:37:03.031723118 +0000 UTC m=+64.599796640 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jjtxg" (UID: "0f504b49-7bd7-4d7e-8de4-bcc282566088") : secret "insights-runtime-extractor-tls" not found Jun 11 18:36:55.031801 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:55.031786 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 18:36:55.031920 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:55.031836 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm podName:0f504b49-7bd7-4d7e-8de4-bcc282566088 nodeName:}" failed. No retries permitted until 2026-06-11 18:37:03.031823343 +0000 UTC m=+64.599896815 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jjtxg" (UID: "0f504b49-7bd7-4d7e-8de4-bcc282566088") : configmap "kube-rbac-proxy" not found Jun 11 18:36:55.312661 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:55.312629 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" event={"ID":"5c322acb-2426-4224-a49a-68ad890205bc","Type":"ContainerStarted","Data":"04ff986abbccc9c0b29b915cf3ccc2a592de8d0e7bdce9cc253290b22ba6ec94"} Jun 11 18:36:56.317899 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:56.317863 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" event={"ID":"5c322acb-2426-4224-a49a-68ad890205bc","Type":"ContainerStarted","Data":"938812e039f49e7c241f047063a20d0ad3bbd56097ba074d4ba80f4145ca9b93"} Jun 11 18:36:56.335924 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:56.335871 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-dgg2s" podStartSLOduration=25.610471696 podStartE2EDuration="27.335853054s" totalImagesPullingTime="1.725381358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:56.334701884 +0000 UTC m=+57.902775365" watchObservedRunningTime="2026-06-11 18:36:56.335853054 +0000 UTC m=+57.903926534" Jun 11 18:36:57.305716 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.305685 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d7cdb9796-wnw7p"] Jun 11 18:36:57.305943 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:36:57.305926 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" podUID="6f7bd047-0864-4400-8fe8-dada6f2e942b" Jun 11 18:36:57.324221 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.324190 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:57.334892 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.334868 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:57.345610 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.345586 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f5b8b6fc8-97jhj"] Jun 11 18:36:57.348483 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.348467 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.365303 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.365276 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f5b8b6fc8-97jhj"] Jun 11 18:36:57.454121 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.454084 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6f7bd047-0864-4400-8fe8-dada6f2e942b-installation-pull-secrets\") pod \"6f7bd047-0864-4400-8fe8-dada6f2e942b\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " Jun 11 18:36:57.454297 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.454140 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-certificates\") pod \"6f7bd047-0864-4400-8fe8-dada6f2e942b\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " Jun 11 18:36:57.454297 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.454202 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6f7bd047-0864-4400-8fe8-dada6f2e942b-ca-trust-extracted\") pod \"6f7bd047-0864-4400-8fe8-dada6f2e942b\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " Jun 11 18:36:57.454297 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.454238 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6f7bd047-0864-4400-8fe8-dada6f2e942b-image-registry-private-configuration\") pod \"6f7bd047-0864-4400-8fe8-dada6f2e942b\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " Jun 11 18:36:57.454297 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.454295 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f7bd047-0864-4400-8fe8-dada6f2e942b-trusted-ca\") pod \"6f7bd047-0864-4400-8fe8-dada6f2e942b\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " Jun 11 18:36:57.454493 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.454324 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp5fx\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-kube-api-access-mp5fx\") pod \"6f7bd047-0864-4400-8fe8-dada6f2e942b\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " Jun 11 18:36:57.454493 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.454415 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-bound-sa-token\") pod \"6f7bd047-0864-4400-8fe8-dada6f2e942b\" (UID: \"6f7bd047-0864-4400-8fe8-dada6f2e942b\") " Jun 11 18:36:57.454596 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.454526 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6f7bd047-0864-4400-8fe8-dada6f2e942b" (UID: "6f7bd047-0864-4400-8fe8-dada6f2e942b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 18:36:57.454668 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.454607 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef47c69c-b12b-4ce7-af1f-84f267eba571-trusted-ca\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.454751 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.454731 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef47c69c-b12b-4ce7-af1f-84f267eba571-installation-pull-secrets\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.455023 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.455007 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef47c69c-b12b-4ce7-af1f-84f267eba571-registry-tls\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.455122 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.454771 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f7bd047-0864-4400-8fe8-dada6f2e942b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6f7bd047-0864-4400-8fe8-dada6f2e942b" (UID: "6f7bd047-0864-4400-8fe8-dada6f2e942b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 18:36:57.455122 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.455108 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef47c69c-b12b-4ce7-af1f-84f267eba571-registry-certificates\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.455320 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.454520 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7bd047-0864-4400-8fe8-dada6f2e942b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6f7bd047-0864-4400-8fe8-dada6f2e942b" (UID: "6f7bd047-0864-4400-8fe8-dada6f2e942b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 18:36:57.455320 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.455164 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef47c69c-b12b-4ce7-af1f-84f267eba571-bound-sa-token\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.455320 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.455259 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hvh7\" (UniqueName: \"kubernetes.io/projected/ef47c69c-b12b-4ce7-af1f-84f267eba571-kube-api-access-8hvh7\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.455320 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.455316 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef47c69c-b12b-4ce7-af1f-84f267eba571-ca-trust-extracted\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.455541 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.455380 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef47c69c-b12b-4ce7-af1f-84f267eba571-image-registry-private-configuration\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.455541 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.455458 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f7bd047-0864-4400-8fe8-dada6f2e942b-trusted-ca\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:36:57.455541 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.455481 2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-certificates\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:36:57.455541 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.455498 2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6f7bd047-0864-4400-8fe8-dada6f2e942b-ca-trust-extracted\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:36:57.456665 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.456642 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f7bd047-0864-4400-8fe8-dada6f2e942b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6f7bd047-0864-4400-8fe8-dada6f2e942b" (UID: "6f7bd047-0864-4400-8fe8-dada6f2e942b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 18:36:57.456773 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.456750 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-kube-api-access-mp5fx" (OuterVolumeSpecName: "kube-api-access-mp5fx") pod "6f7bd047-0864-4400-8fe8-dada6f2e942b" (UID: "6f7bd047-0864-4400-8fe8-dada6f2e942b"). InnerVolumeSpecName "kube-api-access-mp5fx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 18:36:57.456904 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.456878 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f7bd047-0864-4400-8fe8-dada6f2e942b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6f7bd047-0864-4400-8fe8-dada6f2e942b" (UID: "6f7bd047-0864-4400-8fe8-dada6f2e942b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 18:36:57.456904 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.456894 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6f7bd047-0864-4400-8fe8-dada6f2e942b" (UID: "6f7bd047-0864-4400-8fe8-dada6f2e942b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 18:36:57.556899 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.556809 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef47c69c-b12b-4ce7-af1f-84f267eba571-installation-pull-secrets\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.556899 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.556865 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef47c69c-b12b-4ce7-af1f-84f267eba571-registry-tls\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.556899 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.556883 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef47c69c-b12b-4ce7-af1f-84f267eba571-registry-certificates\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.556899 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.556903 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef47c69c-b12b-4ce7-af1f-84f267eba571-bound-sa-token\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.557251 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.556936 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hvh7\" (UniqueName: \"kubernetes.io/projected/ef47c69c-b12b-4ce7-af1f-84f267eba571-kube-api-access-8hvh7\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.557251 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.556965 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef47c69c-b12b-4ce7-af1f-84f267eba571-ca-trust-extracted\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.557251 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.556988 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef47c69c-b12b-4ce7-af1f-84f267eba571-image-registry-private-configuration\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.557251 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.557054 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef47c69c-b12b-4ce7-af1f-84f267eba571-trusted-ca\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.557251 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.557131 2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-bound-sa-token\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:36:57.557251 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.557146 2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6f7bd047-0864-4400-8fe8-dada6f2e942b-installation-pull-secrets\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:36:57.557251 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.557162 2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6f7bd047-0864-4400-8fe8-dada6f2e942b-image-registry-private-configuration\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:36:57.557251 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.557202 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp5fx\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-kube-api-access-mp5fx\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:36:57.557841 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.557817 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef47c69c-b12b-4ce7-af1f-84f267eba571-ca-trust-extracted\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.557904 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.557820 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef47c69c-b12b-4ce7-af1f-84f267eba571-registry-certificates\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.558000 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.557984 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef47c69c-b12b-4ce7-af1f-84f267eba571-trusted-ca\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.559527 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.559502 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef47c69c-b12b-4ce7-af1f-84f267eba571-installation-pull-secrets\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.559862 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.559838 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef47c69c-b12b-4ce7-af1f-84f267eba571-registry-tls\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.559919 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.559846 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef47c69c-b12b-4ce7-af1f-84f267eba571-image-registry-private-configuration\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.565299 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.565275 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef47c69c-b12b-4ce7-af1f-84f267eba571-bound-sa-token\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.565807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.565790 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hvh7\" (UniqueName: \"kubernetes.io/projected/ef47c69c-b12b-4ce7-af1f-84f267eba571-kube-api-access-8hvh7\") pod \"image-registry-6f5b8b6fc8-97jhj\" (UID: \"ef47c69c-b12b-4ce7-af1f-84f267eba571\") " pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.657149 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.657106 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:57.803102 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:57.803070 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f5b8b6fc8-97jhj"] Jun 11 18:36:57.806630 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:36:57.806596 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef47c69c_b12b_4ce7_af1f_84f267eba571.slice/crio-7c6058c2a28eae7e850c20597221f816417216fe03bfe6dd1754c3238aa16e80 WatchSource:0}: Error finding container 7c6058c2a28eae7e850c20597221f816417216fe03bfe6dd1754c3238aa16e80: Status 404 returned error can't find the container with id 7c6058c2a28eae7e850c20597221f816417216fe03bfe6dd1754c3238aa16e80 Jun 11 18:36:58.328717 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:58.328687 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d7cdb9796-wnw7p" Jun 11 18:36:58.328717 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:58.328709 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" event={"ID":"ef47c69c-b12b-4ce7-af1f-84f267eba571","Type":"ContainerStarted","Data":"291aefdcf55680497b614104af51f44f2558ecb21e44e44c493d33f16933227a"} Jun 11 18:36:58.329260 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:58.328755 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" event={"ID":"ef47c69c-b12b-4ce7-af1f-84f267eba571","Type":"ContainerStarted","Data":"7c6058c2a28eae7e850c20597221f816417216fe03bfe6dd1754c3238aa16e80"} Jun 11 18:36:58.329260 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:58.328791 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:36:58.352041 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:58.351995 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" podStartSLOduration=1.351979749 podStartE2EDuration="1.351979749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:36:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:36:58.350923694 +0000 UTC m=+59.918997173" watchObservedRunningTime="2026-06-11 18:36:58.351979749 +0000 UTC m=+59.920053229" Jun 11 18:36:58.393004 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:58.392971 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d7cdb9796-wnw7p"] Jun 11 18:36:58.402521 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:58.402496 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d7cdb9796-wnw7p"] Jun 11 18:36:58.468546 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:58.468313 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6f7bd047-0864-4400-8fe8-dada6f2e942b-registry-tls\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:36:58.917488 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:58.917451 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f7bd047-0864-4400-8fe8-dada6f2e942b" path="/var/lib/kubelet/pods/6f7bd047-0864-4400-8fe8-dada6f2e942b/volumes" Jun 11 18:36:59.912795 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:36:59.912759 2561 scope.go:117] "RemoveContainer" containerID="df0de100bf003c615472461f1450ded54e444f71f41744466dfed9c4a71675a3" Jun 11 18:37:00.337095 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:00.337071 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 18:37:00.337281 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:00.337145 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" event={"ID":"d4de0340-920b-472d-8ff1-0f6b69840a5e","Type":"ContainerStarted","Data":"ca3d98683a1165e82a9915b5d3a6859f9284f3c618c1ed01bccc5907a063c57c"} Jun 11 18:37:00.337453 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:00.337437 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:37:00.342036 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:00.342014 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" Jun 11 18:37:00.361932 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:00.361887 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5dbbb5d744-kpstz" podStartSLOduration=24.105255943 podStartE2EDuration="31.361871218s" totalImagesPullingTime="7.256615275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:00.361498001 +0000 UTC m=+61.929571482" watchObservedRunningTime="2026-06-11 18:37:00.361871218 +0000 UTC m=+61.929944698" Jun 11 18:37:02.206075 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:02.206048 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhtcz" Jun 11 18:37:03.126131 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:03.126073 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:37:03.126323 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:03.126248 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:37:03.126824 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:03.126796 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0f504b49-7bd7-4d7e-8de4-bcc282566088-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:37:03.128508 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:03.128489 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0f504b49-7bd7-4d7e-8de4-bcc282566088-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjtxg\" (UID: \"0f504b49-7bd7-4d7e-8de4-bcc282566088\") " pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:37:03.241284 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:03.241253 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rbv24\"" Jun 11 18:37:03.250026 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:03.250006 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jjtxg" Jun 11 18:37:03.379005 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:03.378943 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jjtxg"] Jun 11 18:37:03.381528 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:37:03.381496 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f504b49_7bd7_4d7e_8de4_bcc282566088.slice/crio-6e875dc814a2ae2f17674c97bbd1eb1700038aaeaffb4948d7154b1db39ecb4f WatchSource:0}: Error finding container 6e875dc814a2ae2f17674c97bbd1eb1700038aaeaffb4948d7154b1db39ecb4f: Status 404 returned error can't find the container with id 6e875dc814a2ae2f17674c97bbd1eb1700038aaeaffb4948d7154b1db39ecb4f Jun 11 18:37:04.353220 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:04.353166 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jjtxg" event={"ID":"0f504b49-7bd7-4d7e-8de4-bcc282566088","Type":"ContainerStarted","Data":"6e875dc814a2ae2f17674c97bbd1eb1700038aaeaffb4948d7154b1db39ecb4f"} Jun 11 18:37:04.640119 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:04.640035 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs\") pod \"network-metrics-daemon-fjvnc\" (UID: \"0a9668ae-c995-40a8-b7fd-de034a8fff15\") " pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:37:04.642485 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:04.642455 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 18:37:04.653400 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:04.653371 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a9668ae-c995-40a8-b7fd-de034a8fff15-metrics-certs\") pod \"network-metrics-daemon-fjvnc\" (UID: \"0a9668ae-c995-40a8-b7fd-de034a8fff15\") " pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:37:04.741042 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:04.741001 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:37:04.743471 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:04.743446 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 18:37:04.753984 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:04.753957 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb731d8a-734b-48f1-87a2-303368226673-original-pull-secret\") pod \"global-pull-secret-syncer-58lbd\" (UID: \"fb731d8a-734b-48f1-87a2-303368226673\") " pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:37:04.841597 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:04.841561 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frv7t\" (UniqueName: \"kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t\") pod \"network-check-target-dk499\" (UID: \"67b90c12-c8b0-4ceb-a8df-8369f7fff5a0\") " pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:37:04.844464 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:04.844435 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frv7t\" (UniqueName: \"kubernetes.io/projected/67b90c12-c8b0-4ceb-a8df-8369f7fff5a0-kube-api-access-frv7t\") pod \"network-check-target-dk499\" (UID: \"67b90c12-c8b0-4ceb-a8df-8369f7fff5a0\") " pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:37:04.923928 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:04.923862 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58lbd" Jun 11 18:37:04.932222 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:04.932193 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ptdz8\"" Jun 11 18:37:04.936665 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:04.936640 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qf6vw\"" Jun 11 18:37:04.940453 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:04.940430 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:37:04.945788 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:04.945768 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjvnc" Jun 11 18:37:07.056271 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:07.056237 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fjvnc"] Jun 11 18:37:07.073864 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:07.073836 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-58lbd"] Jun 11 18:37:07.077501 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:37:07.077474 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb731d8a_734b_48f1_87a2_303368226673.slice/crio-a1f43682cb0aac9977dd5b5d6faa9e5051d1b985a7219faa299d14f181071cdb WatchSource:0}: Error finding container a1f43682cb0aac9977dd5b5d6faa9e5051d1b985a7219faa299d14f181071cdb: Status 404 returned error can't find the container with id a1f43682cb0aac9977dd5b5d6faa9e5051d1b985a7219faa299d14f181071cdb Jun 11 18:37:07.090810 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:07.090782 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dk499"] Jun 11 18:37:07.093889 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:37:07.093851 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b90c12_c8b0_4ceb_a8df_8369f7fff5a0.slice/crio-d0ae00a79773a34700b3a53bff24c0a4231ba31dd868cb570b3b544588747e00 WatchSource:0}: Error finding container d0ae00a79773a34700b3a53bff24c0a4231ba31dd868cb570b3b544588747e00: Status 404 returned error can't find the container with id d0ae00a79773a34700b3a53bff24c0a4231ba31dd868cb570b3b544588747e00 Jun 11 18:37:07.365244 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:07.365205 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-58lbd" event={"ID":"fb731d8a-734b-48f1-87a2-303368226673","Type":"ContainerStarted","Data":"a1f43682cb0aac9977dd5b5d6faa9e5051d1b985a7219faa299d14f181071cdb"} Jun 11 18:37:07.366367 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:07.366344 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fjvnc" event={"ID":"0a9668ae-c995-40a8-b7fd-de034a8fff15","Type":"ContainerStarted","Data":"a6177dee31ef4fc229f278a6717f66f0fa64d4576dea2029d429f037869dbdaf"} Jun 11 18:37:07.367858 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:07.367833 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dk499" event={"ID":"67b90c12-c8b0-4ceb-a8df-8369f7fff5a0","Type":"ContainerStarted","Data":"54922181b3c231ed1ec83e43b37ef49d138eda37a56b1660652ffa34a8c8d842"} Jun 11 18:37:07.367974 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:07.367870 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dk499" event={"ID":"67b90c12-c8b0-4ceb-a8df-8369f7fff5a0","Type":"ContainerStarted","Data":"d0ae00a79773a34700b3a53bff24c0a4231ba31dd868cb570b3b544588747e00"} Jun 11 18:37:07.368038 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:07.367973 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:37:07.369262 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:07.369237 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jjtxg" event={"ID":"0f504b49-7bd7-4d7e-8de4-bcc282566088","Type":"ContainerStarted","Data":"71498f30ec870c8ec56cca0fc58eb647d3803a8b381383437bccf5253321305d"} Jun 11 18:37:07.383303 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:07.383254 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dk499" podStartSLOduration=68.383238237 podStartE2EDuration="1m8.383238237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:35:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:07.382831943 +0000 UTC m=+68.950905451" watchObservedRunningTime="2026-06-11 18:37:07.383238237 +0000 UTC m=+68.951311779" Jun 11 18:37:08.376405 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:08.376366 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fjvnc" event={"ID":"0a9668ae-c995-40a8-b7fd-de034a8fff15","Type":"ContainerStarted","Data":"aae31ac550c364e624c3eec08a4fd9e892e16da6bf3f19016621f132e004033d"} Jun 11 18:37:08.376876 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:08.376434 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fjvnc" event={"ID":"0a9668ae-c995-40a8-b7fd-de034a8fff15","Type":"ContainerStarted","Data":"2559525ebd20af35cdee9789b78c236dc771efea375225e4394c6b94586eaa1f"} Jun 11 18:37:08.378474 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:08.378440 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jjtxg" event={"ID":"0f504b49-7bd7-4d7e-8de4-bcc282566088","Type":"ContainerStarted","Data":"f962c1e1091794772a7b6c24cc9649cb268472a408ff04967273d03727ad8251"} Jun 11 18:37:08.392975 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:08.392925 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fjvnc" podStartSLOduration=68.431276828 podStartE2EDuration="1m9.392906499s" totalImagesPullingTime="961.629671ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:08.392886095 +0000 UTC m=+69.960959575" watchObservedRunningTime="2026-06-11 18:37:08.392906499 +0000 UTC m=+69.960979980" Jun 11 18:37:09.284346 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.284305 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:37:09.284530 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.284364 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-szglk\" (UID: \"3745889d-fbde-41e9-b083-3a33484ded3a\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" Jun 11 18:37:09.284530 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.284395 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:37:09.284530 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.284432 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:37:09.284530 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.284473 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-85bbt\" (UID: \"08e48b3b-9d88-4715-aa6a-008e8cdebb3e\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:37:09.285153 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.285101 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1651923-19d8-41b4-8f23-21afb8ac43a7-service-ca-bundle\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:37:09.287636 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.287583 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1651923-19d8-41b4-8f23-21afb8ac43a7-metrics-certs\") pod \"router-default-85b5fb789b-jt8gg\" (UID: \"c1651923-19d8-41b4-8f23-21afb8ac43a7\") " pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:37:09.287775 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.287663 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e42d52fc-e45a-415f-9694-26853b83c2be-metrics-tls\") pod \"dns-default-pxnqh\" (UID: \"e42d52fc-e45a-415f-9694-26853b83c2be\") " pod="openshift-dns/dns-default-pxnqh" Jun 11 18:37:09.287868 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.287844 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3745889d-fbde-41e9-b083-3a33484ded3a-networking-console-plugin-cert\") pod \"networking-console-plugin-867b76cb74-szglk\" (UID: \"3745889d-fbde-41e9-b083-3a33484ded3a\") " pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" Jun 11 18:37:09.288371 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.288348 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/08e48b3b-9d88-4715-aa6a-008e8cdebb3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-f9d7df769-85bbt\" (UID: \"08e48b3b-9d88-4715-aa6a-008e8cdebb3e\") " pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:37:09.406776 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.406748 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nhg9x\"" Jun 11 18:37:09.414757 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.414729 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pxnqh" Jun 11 18:37:09.426590 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.426563 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2hjj6\"" Jun 11 18:37:09.434473 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.434443 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" Jun 11 18:37:09.471009 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.470976 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-55k76\"" Jun 11 18:37:09.479198 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.479153 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" Jun 11 18:37:09.482986 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.482965 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-sk7rt\"" Jun 11 18:37:09.490978 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:09.490952 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:37:11.399441 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:11.399006 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jjtxg" event={"ID":"0f504b49-7bd7-4d7e-8de4-bcc282566088","Type":"ContainerStarted","Data":"dcb51107453ad6af9699b2c15d3cda8c1d242cb78442ccc1248d15905a99b7c3"} Jun 11 18:37:11.424907 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:11.424854 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jjtxg" podStartSLOduration=16.732060409 podStartE2EDuration="24.42483631s" totalImagesPullingTime="7.692775901s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 18:36:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:11.423474021 +0000 UTC m=+72.991547531" watchObservedRunningTime="2026-06-11 18:37:11.42483631 +0000 UTC m=+72.992909790" Jun 11 18:37:11.444024 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:11.443934 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt"] Jun 11 18:37:11.445499 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:11.445469 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pxnqh"] Jun 11 18:37:11.446234 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:37:11.446203 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42d52fc_e45a_415f_9694_26853b83c2be.slice/crio-1031fb7f6ef4fc0419e995dbeb81325aef12151ea0ea5eb4d760ae37ed0c054d WatchSource:0}: Error finding container 1031fb7f6ef4fc0419e995dbeb81325aef12151ea0ea5eb4d760ae37ed0c054d: Status 404 returned error can't find the container with id 1031fb7f6ef4fc0419e995dbeb81325aef12151ea0ea5eb4d760ae37ed0c054d Jun 11 18:37:11.447292 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:37:11.447246 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08e48b3b_9d88_4715_aa6a_008e8cdebb3e.slice/crio-695a24e6be2f8fa6074a9dfa04f5acd9e319924379cf3abf8be55abc75a21fd9 WatchSource:0}: Error finding container 695a24e6be2f8fa6074a9dfa04f5acd9e319924379cf3abf8be55abc75a21fd9: Status 404 returned error can't find the container with id 695a24e6be2f8fa6074a9dfa04f5acd9e319924379cf3abf8be55abc75a21fd9 Jun 11 18:37:11.681933 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:11.681910 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-867b76cb74-szglk"] Jun 11 18:37:11.684603 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:37:11.684570 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3745889d_fbde_41e9_b083_3a33484ded3a.slice/crio-c765d49f8e4e4f2ee9d41b156395d584cff654af13bdef94b4de3e7c11def205 WatchSource:0}: Error finding container c765d49f8e4e4f2ee9d41b156395d584cff654af13bdef94b4de3e7c11def205: Status 404 returned error can't find the container with id c765d49f8e4e4f2ee9d41b156395d584cff654af13bdef94b4de3e7c11def205 Jun 11 18:37:11.684728 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:11.684621 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-85b5fb789b-jt8gg"] Jun 11 18:37:11.687807 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:37:11.687783 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1651923_19d8_41b4_8f23_21afb8ac43a7.slice/crio-c4030fc07769cf8959afa69f4d92f8f47dc2878f37b4c35626476014b4803418 WatchSource:0}: Error finding container c4030fc07769cf8959afa69f4d92f8f47dc2878f37b4c35626476014b4803418: Status 404 returned error can't find the container with id c4030fc07769cf8959afa69f4d92f8f47dc2878f37b4c35626476014b4803418 Jun 11 18:37:12.406562 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:12.406522 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" event={"ID":"c1651923-19d8-41b4-8f23-21afb8ac43a7","Type":"ContainerStarted","Data":"b3974a555caca2c1d2ea4f970145478ba2812f5bb18ceb5d6bc5c478f20f6a49"} Jun 11 18:37:12.407066 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:12.406584 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" event={"ID":"c1651923-19d8-41b4-8f23-21afb8ac43a7","Type":"ContainerStarted","Data":"c4030fc07769cf8959afa69f4d92f8f47dc2878f37b4c35626476014b4803418"} Jun 11 18:37:12.408382 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:12.408352 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" event={"ID":"08e48b3b-9d88-4715-aa6a-008e8cdebb3e","Type":"ContainerStarted","Data":"695a24e6be2f8fa6074a9dfa04f5acd9e319924379cf3abf8be55abc75a21fd9"} Jun 11 18:37:12.409963 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:12.409825 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pxnqh" event={"ID":"e42d52fc-e45a-415f-9694-26853b83c2be","Type":"ContainerStarted","Data":"1031fb7f6ef4fc0419e995dbeb81325aef12151ea0ea5eb4d760ae37ed0c054d"} Jun 11 18:37:12.412265 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:12.412217 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-58lbd" event={"ID":"fb731d8a-734b-48f1-87a2-303368226673","Type":"ContainerStarted","Data":"29b6954482090dfa33675c9404f676897a034744c9d61abbcfca1fa34f29d743"} Jun 11 18:37:12.413904 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:12.413857 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" event={"ID":"3745889d-fbde-41e9-b083-3a33484ded3a","Type":"ContainerStarted","Data":"c765d49f8e4e4f2ee9d41b156395d584cff654af13bdef94b4de3e7c11def205"} Jun 11 18:37:12.428879 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:12.428728 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" podStartSLOduration=43.428710845 podStartE2EDuration="43.428710845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:36:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:12.427570304 +0000 UTC m=+73.995643804" watchObservedRunningTime="2026-06-11 18:37:12.428710845 +0000 UTC m=+73.996784326" Jun 11 18:37:12.443742 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:12.443693 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-58lbd" podStartSLOduration=69.243159633 podStartE2EDuration="1m13.443676627s" totalImagesPullingTime="4.200516994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 18:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:12.443057753 +0000 UTC m=+74.011131234" watchObservedRunningTime="2026-06-11 18:37:12.443676627 +0000 UTC m=+74.011750115" Jun 11 18:37:12.491427 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:12.491386 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:37:12.494481 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:12.494447 2561 patch_prober.go:28] interesting pod/router-default-85b5fb789b-jt8gg container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jun 11 18:37:12.494481 ip-10-0-138-156 kubenswrapper[2561]: [-]has-synced failed: reason withheld Jun 11 18:37:12.494481 ip-10-0-138-156 kubenswrapper[2561]: [+]process-running ok Jun 11 18:37:12.494481 ip-10-0-138-156 kubenswrapper[2561]: healthz check failed Jun 11 18:37:12.494746 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:12.494515 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" podUID="c1651923-19d8-41b4-8f23-21afb8ac43a7" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 18:37:13.492780 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:13.492743 2561 patch_prober.go:28] interesting pod/router-default-85b5fb789b-jt8gg container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jun 11 18:37:13.492780 ip-10-0-138-156 kubenswrapper[2561]: [-]has-synced failed: reason withheld Jun 11 18:37:13.492780 ip-10-0-138-156 kubenswrapper[2561]: [+]process-running ok Jun 11 18:37:13.492780 ip-10-0-138-156 kubenswrapper[2561]: healthz check failed Jun 11 18:37:13.493254 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:13.492803 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" podUID="c1651923-19d8-41b4-8f23-21afb8ac43a7" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 18:37:14.492492 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:14.492459 2561 patch_prober.go:28] interesting pod/router-default-85b5fb789b-jt8gg container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jun 11 18:37:14.492492 ip-10-0-138-156 kubenswrapper[2561]: [-]has-synced failed: reason withheld Jun 11 18:37:14.492492 ip-10-0-138-156 kubenswrapper[2561]: [+]process-running ok Jun 11 18:37:14.492492 ip-10-0-138-156 kubenswrapper[2561]: healthz check failed Jun 11 18:37:14.492714 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:14.492515 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" podUID="c1651923-19d8-41b4-8f23-21afb8ac43a7" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 18:37:15.426426 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:15.426399 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" event={"ID":"08e48b3b-9d88-4715-aa6a-008e8cdebb3e","Type":"ContainerStarted","Data":"95fb087e5dc9e44c0af4e6bfd3b0a3229d877886b17c93cba161af3c7cec2884"} Jun 11 18:37:15.428165 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:15.428143 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pxnqh" event={"ID":"e42d52fc-e45a-415f-9694-26853b83c2be","Type":"ContainerStarted","Data":"ab04817ecd69041cf8dffe328c4f8df851c9e10834b352af0023b7c792856f1b"} Jun 11 18:37:15.428275 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:15.428207 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pxnqh" Jun 11 18:37:15.428275 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:15.428233 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pxnqh" event={"ID":"e42d52fc-e45a-415f-9694-26853b83c2be","Type":"ContainerStarted","Data":"373b5788a700da5b32368f964f74f7f65ced098b742e6f4dac2ce9430a79e163"} Jun 11 18:37:15.429582 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:15.429561 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" event={"ID":"3745889d-fbde-41e9-b083-3a33484ded3a","Type":"ContainerStarted","Data":"5eec28f4ffff369cbc20cc86eabfc94c2092162d6825024a4f4c812056e08b7b"} Jun 11 18:37:15.446161 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:15.446116 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-f9d7df769-85bbt" podStartSLOduration=43.352033344 podStartE2EDuration="46.446103979s" totalImagesPullingTime="3.094070635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:15.44513873 +0000 UTC m=+77.013212209" watchObservedRunningTime="2026-06-11 18:37:15.446103979 +0000 UTC m=+77.014177482" Jun 11 18:37:15.461934 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:15.461890 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pxnqh" podStartSLOduration=35.362056823 podStartE2EDuration="38.46187638s" totalImagesPullingTime="3.099819557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:15.460618194 +0000 UTC m=+77.028691674" watchObservedRunningTime="2026-06-11 18:37:15.46187638 +0000 UTC m=+77.029949859" Jun 11 18:37:15.475887 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:15.475848 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-867b76cb74-szglk" podStartSLOduration=43.618107408 podStartE2EDuration="46.475835478s" totalImagesPullingTime="2.85772807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:15.474710604 +0000 UTC m=+77.042784107" watchObservedRunningTime="2026-06-11 18:37:15.475835478 +0000 UTC m=+77.043908958" Jun 11 18:37:15.492070 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:15.492046 2561 patch_prober.go:28] interesting pod/router-default-85b5fb789b-jt8gg container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jun 11 18:37:15.492070 ip-10-0-138-156 kubenswrapper[2561]: [-]has-synced failed: reason withheld Jun 11 18:37:15.492070 ip-10-0-138-156 kubenswrapper[2561]: [+]process-running ok Jun 11 18:37:15.492070 ip-10-0-138-156 kubenswrapper[2561]: healthz check failed Jun 11 18:37:15.492248 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:15.492096 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" podUID="c1651923-19d8-41b4-8f23-21afb8ac43a7" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 18:37:16.492242 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:16.492199 2561 patch_prober.go:28] interesting pod/router-default-85b5fb789b-jt8gg container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jun 11 18:37:16.492242 ip-10-0-138-156 kubenswrapper[2561]: [-]has-synced failed: reason withheld Jun 11 18:37:16.492242 ip-10-0-138-156 kubenswrapper[2561]: [+]process-running ok Jun 11 18:37:16.492242 ip-10-0-138-156 kubenswrapper[2561]: healthz check failed Jun 11 18:37:16.492701 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:16.492274 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" podUID="c1651923-19d8-41b4-8f23-21afb8ac43a7" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 18:37:17.492314 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:17.492277 2561 patch_prober.go:28] interesting pod/router-default-85b5fb789b-jt8gg container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jun 11 18:37:17.492314 ip-10-0-138-156 kubenswrapper[2561]: [-]has-synced failed: reason withheld Jun 11 18:37:17.492314 ip-10-0-138-156 kubenswrapper[2561]: [+]process-running ok Jun 11 18:37:17.492314 ip-10-0-138-156 kubenswrapper[2561]: healthz check failed Jun 11 18:37:17.492789 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:17.492349 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" podUID="c1651923-19d8-41b4-8f23-21afb8ac43a7" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 18:37:17.661918 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:17.661880 2561 patch_prober.go:28] interesting pod/image-registry-6f5b8b6fc8-97jhj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 18:37:17.662082 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:17.661936 2561 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" podUID="ef47c69c-b12b-4ce7-af1f-84f267eba571" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 18:37:18.492534 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:18.492497 2561 patch_prober.go:28] interesting pod/router-default-85b5fb789b-jt8gg container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jun 11 18:37:18.492534 ip-10-0-138-156 kubenswrapper[2561]: [-]has-synced failed: reason withheld Jun 11 18:37:18.492534 ip-10-0-138-156 kubenswrapper[2561]: [+]process-running ok Jun 11 18:37:18.492534 ip-10-0-138-156 kubenswrapper[2561]: healthz check failed Jun 11 18:37:18.492990 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:18.492553 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" podUID="c1651923-19d8-41b4-8f23-21afb8ac43a7" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 18:37:19.336954 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:19.336921 2561 patch_prober.go:28] interesting pod/image-registry-6f5b8b6fc8-97jhj container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 18:37:19.337126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:19.336973 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" podUID="ef47c69c-b12b-4ce7-af1f-84f267eba571" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 18:37:19.491502 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:19.491460 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:37:19.492597 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:19.492566 2561 patch_prober.go:28] interesting pod/router-default-85b5fb789b-jt8gg container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jun 11 18:37:19.492597 ip-10-0-138-156 kubenswrapper[2561]: [-]has-synced failed: reason withheld Jun 11 18:37:19.492597 ip-10-0-138-156 kubenswrapper[2561]: [+]process-running ok Jun 11 18:37:19.492597 ip-10-0-138-156 kubenswrapper[2561]: healthz check failed Jun 11 18:37:19.493008 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:19.492614 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" podUID="c1651923-19d8-41b4-8f23-21afb8ac43a7" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 18:37:20.492900 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:20.492865 2561 patch_prober.go:28] interesting pod/router-default-85b5fb789b-jt8gg container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jun 11 18:37:20.492900 ip-10-0-138-156 kubenswrapper[2561]: [-]has-synced failed: reason withheld Jun 11 18:37:20.492900 ip-10-0-138-156 kubenswrapper[2561]: [+]process-running ok Jun 11 18:37:20.492900 ip-10-0-138-156 kubenswrapper[2561]: healthz check failed Jun 11 18:37:20.493373 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:20.492920 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" podUID="c1651923-19d8-41b4-8f23-21afb8ac43a7" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 18:37:21.492390 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:21.492352 2561 patch_prober.go:28] interesting pod/router-default-85b5fb789b-jt8gg container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jun 11 18:37:21.492390 ip-10-0-138-156 kubenswrapper[2561]: [-]has-synced failed: reason withheld Jun 11 18:37:21.492390 ip-10-0-138-156 kubenswrapper[2561]: [+]process-running ok Jun 11 18:37:21.492390 ip-10-0-138-156 kubenswrapper[2561]: healthz check failed Jun 11 18:37:21.492634 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:21.492407 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" podUID="c1651923-19d8-41b4-8f23-21afb8ac43a7" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 18:37:22.492203 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:22.492153 2561 patch_prober.go:28] interesting pod/router-default-85b5fb789b-jt8gg container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jun 11 18:37:22.492203 ip-10-0-138-156 kubenswrapper[2561]: [-]has-synced failed: reason withheld Jun 11 18:37:22.492203 ip-10-0-138-156 kubenswrapper[2561]: [+]process-running ok Jun 11 18:37:22.492203 ip-10-0-138-156 kubenswrapper[2561]: healthz check failed Jun 11 18:37:22.492711 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:22.492226 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" podUID="c1651923-19d8-41b4-8f23-21afb8ac43a7" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 18:37:23.492134 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:23.492096 2561 patch_prober.go:28] interesting pod/router-default-85b5fb789b-jt8gg container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jun 11 18:37:23.492134 ip-10-0-138-156 kubenswrapper[2561]: [-]has-synced failed: reason withheld Jun 11 18:37:23.492134 ip-10-0-138-156 kubenswrapper[2561]: [+]process-running ok Jun 11 18:37:23.492134 ip-10-0-138-156 kubenswrapper[2561]: healthz check failed Jun 11 18:37:23.492595 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:23.492148 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" podUID="c1651923-19d8-41b4-8f23-21afb8ac43a7" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 18:37:24.492973 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:24.492941 2561 patch_prober.go:28] interesting pod/router-default-85b5fb789b-jt8gg container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jun 11 18:37:24.492973 ip-10-0-138-156 kubenswrapper[2561]: [-]has-synced failed: reason withheld Jun 11 18:37:24.492973 ip-10-0-138-156 kubenswrapper[2561]: [+]process-running ok Jun 11 18:37:24.492973 ip-10-0-138-156 kubenswrapper[2561]: healthz check failed Jun 11 18:37:24.493441 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:24.492998 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" podUID="c1651923-19d8-41b4-8f23-21afb8ac43a7" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 18:37:25.436977 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:25.436947 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pxnqh" Jun 11 18:37:25.492709 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:25.492675 2561 patch_prober.go:28] interesting pod/router-default-85b5fb789b-jt8gg container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jun 11 18:37:25.492709 ip-10-0-138-156 kubenswrapper[2561]: [-]has-synced failed: reason withheld Jun 11 18:37:25.492709 ip-10-0-138-156 kubenswrapper[2561]: [+]process-running ok Jun 11 18:37:25.492709 ip-10-0-138-156 kubenswrapper[2561]: healthz check failed Jun 11 18:37:25.492953 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:25.492730 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" podUID="c1651923-19d8-41b4-8f23-21afb8ac43a7" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 18:37:26.493026 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:26.492998 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:37:26.494286 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:26.494266 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-85b5fb789b-jt8gg" Jun 11 18:37:27.661052 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:27.661019 2561 patch_prober.go:28] interesting pod/image-registry-6f5b8b6fc8-97jhj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 18:37:27.661464 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:27.661076 2561 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" podUID="ef47c69c-b12b-4ce7-af1f-84f267eba571" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 18:37:28.261783 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.261750 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx"] Jun 11 18:37:28.269217 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.269189 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.273121 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.273097 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 11 18:37:28.274275 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.274251 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 11 18:37:28.274495 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.274472 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 11 18:37:28.274707 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.274680 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 11 18:37:28.275137 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.275120 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 11 18:37:28.275513 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.275343 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 11 18:37:28.275707 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.275351 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 11 18:37:28.276387 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.276368 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j"] Jun 11 18:37:28.282137 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.282116 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8"] Jun 11 18:37:28.282364 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.282349 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 18:37:28.284845 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.284826 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 11 18:37:28.286711 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.286682 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx"] Jun 11 18:37:28.286814 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.286792 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" Jun 11 18:37:28.289281 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.289260 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-6s9xn\"" Jun 11 18:37:28.289508 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.289413 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 11 18:37:28.305543 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.305409 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j"] Jun 11 18:37:28.334933 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.334896 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtjj8\" (UniqueName: \"kubernetes.io/projected/21ff8dcc-639e-4865-af63-8f609ed3544e-kube-api-access-gtjj8\") pod \"managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8\" (UID: \"21ff8dcc-639e-4865-af63-8f609ed3544e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" Jun 11 18:37:28.335075 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.335026 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7805e125-ba9b-4713-ac48-3678fba0895d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.335148 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.335069 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn5sm\" (UniqueName: \"kubernetes.io/projected/7805e125-ba9b-4713-ac48-3678fba0895d-kube-api-access-gn5sm\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.335148 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.335100 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-hub\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.335148 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.335120 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/21ff8dcc-639e-4865-af63-8f609ed3544e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8\" (UID: \"21ff8dcc-639e-4865-af63-8f609ed3544e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" Jun 11 18:37:28.335148 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.335141 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.335339 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.335158 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/828f0479-53ba-4372-a7b6-108bc6b8355b-klusterlet-config\") pod \"klusterlet-addon-workmgr-7df8d85848-xs64j\" (UID: \"828f0479-53ba-4372-a7b6-108bc6b8355b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 18:37:28.335339 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.335209 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.335339 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.335235 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/828f0479-53ba-4372-a7b6-108bc6b8355b-tmp\") pod \"klusterlet-addon-workmgr-7df8d85848-xs64j\" (UID: \"828f0479-53ba-4372-a7b6-108bc6b8355b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 18:37:28.335339 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.335292 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ws9n\" (UniqueName: \"kubernetes.io/projected/828f0479-53ba-4372-a7b6-108bc6b8355b-kube-api-access-5ws9n\") pod \"klusterlet-addon-workmgr-7df8d85848-xs64j\" (UID: \"828f0479-53ba-4372-a7b6-108bc6b8355b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 18:37:28.335467 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.335346 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-ca\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.335930 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.335913 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8"] Jun 11 18:37:28.366947 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.366910 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-kqcj8"] Jun 11 18:37:28.370897 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.370873 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57b6dbc6d6-qn8wb"] Jun 11 18:37:28.371042 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.370991 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-kqcj8" Jun 11 18:37:28.374516 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.374496 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-694c955b9-9xh9f"] Jun 11 18:37:28.374641 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.374613 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.375844 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.375828 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-xb8w9\"" Jun 11 18:37:28.375941 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.375878 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 11 18:37:28.377149 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.377127 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 11 18:37:28.377283 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.377210 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 11 18:37:28.377464 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.377449 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-h4xrm\"" Jun 11 18:37:28.377464 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.377462 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 11 18:37:28.377543 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.377518 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 11 18:37:28.377669 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.377657 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 11 18:37:28.378126 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.378113 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-694c955b9-9xh9f" Jun 11 18:37:28.378161 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.378145 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 11 18:37:28.378245 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.378232 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 11 18:37:28.387925 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.387909 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-9n6zc\"" Jun 11 18:37:28.404725 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.404695 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-694c955b9-9xh9f"] Jun 11 18:37:28.407301 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.407279 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-kqcj8"] Jun 11 18:37:28.408877 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.408818 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57b6dbc6d6-qn8wb"] Jun 11 18:37:28.435758 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.435727 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-oauth-config\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.435899 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.435769 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6xsz\" (UniqueName: \"kubernetes.io/projected/f6096325-84a3-4c44-ba18-b4d74e5dd244-kube-api-access-m6xsz\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.435899 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.435798 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-config\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.435899 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.435842 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-service-ca\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.435995 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.435961 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7805e125-ba9b-4713-ac48-3678fba0895d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.436030 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436000 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3f9034ca-6ad4-4c91-8103-55ebb839277f-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-kqcj8\" (UID: \"3f9034ca-6ad4-4c91-8103-55ebb839277f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-kqcj8" Jun 11 18:37:28.436073 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436028 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-serving-cert\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.436129 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436098 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-oauth-serving-cert\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.436202 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436149 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn5sm\" (UniqueName: \"kubernetes.io/projected/7805e125-ba9b-4713-ac48-3678fba0895d-kube-api-access-gn5sm\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.436264 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436218 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-hub\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.436264 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436248 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6n22\" (UniqueName: \"kubernetes.io/projected/84d86e1c-4a2f-4568-aaf1-d14b11521f53-kube-api-access-g6n22\") pod \"downloads-694c955b9-9xh9f\" (UID: \"84d86e1c-4a2f-4568-aaf1-d14b11521f53\") " pod="openshift-console/downloads-694c955b9-9xh9f" Jun 11 18:37:28.436372 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436279 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/21ff8dcc-639e-4865-af63-8f609ed3544e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8\" (UID: \"21ff8dcc-639e-4865-af63-8f609ed3544e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" Jun 11 18:37:28.436372 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436305 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.436372 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436332 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/828f0479-53ba-4372-a7b6-108bc6b8355b-klusterlet-config\") pod \"klusterlet-addon-workmgr-7df8d85848-xs64j\" (UID: \"828f0479-53ba-4372-a7b6-108bc6b8355b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 18:37:28.436372 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436358 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.436557 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436387 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/828f0479-53ba-4372-a7b6-108bc6b8355b-tmp\") pod \"klusterlet-addon-workmgr-7df8d85848-xs64j\" (UID: \"828f0479-53ba-4372-a7b6-108bc6b8355b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 18:37:28.436557 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436425 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ws9n\" (UniqueName: \"kubernetes.io/projected/828f0479-53ba-4372-a7b6-108bc6b8355b-kube-api-access-5ws9n\") pod \"klusterlet-addon-workmgr-7df8d85848-xs64j\" (UID: \"828f0479-53ba-4372-a7b6-108bc6b8355b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 18:37:28.436557 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436464 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-ca\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.436557 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436497 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtjj8\" (UniqueName: \"kubernetes.io/projected/21ff8dcc-639e-4865-af63-8f609ed3544e-kube-api-access-gtjj8\") pod \"managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8\" (UID: \"21ff8dcc-639e-4865-af63-8f609ed3544e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" Jun 11 18:37:28.436778 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436750 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7805e125-ba9b-4713-ac48-3678fba0895d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.436964 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.436925 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/828f0479-53ba-4372-a7b6-108bc6b8355b-tmp\") pod \"klusterlet-addon-workmgr-7df8d85848-xs64j\" (UID: \"828f0479-53ba-4372-a7b6-108bc6b8355b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 18:37:28.439035 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.439013 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-ca\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.439800 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.439752 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-hub\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.439800 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.439780 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.439971 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.439909 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.440030 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.439989 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/828f0479-53ba-4372-a7b6-108bc6b8355b-klusterlet-config\") pod \"klusterlet-addon-workmgr-7df8d85848-xs64j\" (UID: \"828f0479-53ba-4372-a7b6-108bc6b8355b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 18:37:28.440093 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.440024 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/21ff8dcc-639e-4865-af63-8f609ed3544e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8\" (UID: \"21ff8dcc-639e-4865-af63-8f609ed3544e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" Jun 11 18:37:28.452869 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.452848 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn5sm\" (UniqueName: \"kubernetes.io/projected/7805e125-ba9b-4713-ac48-3678fba0895d-kube-api-access-gn5sm\") pod \"cluster-proxy-proxy-agent-779bf9c455-ldghx\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.456096 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.456070 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ws9n\" (UniqueName: \"kubernetes.io/projected/828f0479-53ba-4372-a7b6-108bc6b8355b-kube-api-access-5ws9n\") pod \"klusterlet-addon-workmgr-7df8d85848-xs64j\" (UID: \"828f0479-53ba-4372-a7b6-108bc6b8355b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 18:37:28.457537 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.457513 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtjj8\" (UniqueName: \"kubernetes.io/projected/21ff8dcc-639e-4865-af63-8f609ed3544e-kube-api-access-gtjj8\") pod \"managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8\" (UID: \"21ff8dcc-639e-4865-af63-8f609ed3544e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" Jun 11 18:37:28.537854 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.537768 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3f9034ca-6ad4-4c91-8103-55ebb839277f-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-kqcj8\" (UID: \"3f9034ca-6ad4-4c91-8103-55ebb839277f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-kqcj8" Jun 11 18:37:28.537854 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.537804 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-serving-cert\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.537854 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.537831 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-oauth-serving-cert\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.537854 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.537857 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6n22\" (UniqueName: \"kubernetes.io/projected/84d86e1c-4a2f-4568-aaf1-d14b11521f53-kube-api-access-g6n22\") pod \"downloads-694c955b9-9xh9f\" (UID: \"84d86e1c-4a2f-4568-aaf1-d14b11521f53\") " pod="openshift-console/downloads-694c955b9-9xh9f" Jun 11 18:37:28.538222 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.537910 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-oauth-config\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.538222 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.537934 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6xsz\" (UniqueName: \"kubernetes.io/projected/f6096325-84a3-4c44-ba18-b4d74e5dd244-kube-api-access-m6xsz\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.538222 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.537960 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-config\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.538222 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.537982 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-service-ca\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.538637 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.538614 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-oauth-serving-cert\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.538699 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.538661 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-service-ca\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.538793 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.538774 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-config\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.540765 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.540734 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-oauth-config\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.540861 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.540790 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-serving-cert\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.540902 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.540881 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3f9034ca-6ad4-4c91-8103-55ebb839277f-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-kqcj8\" (UID: \"3f9034ca-6ad4-4c91-8103-55ebb839277f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-kqcj8" Jun 11 18:37:28.549269 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.549248 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6xsz\" (UniqueName: \"kubernetes.io/projected/f6096325-84a3-4c44-ba18-b4d74e5dd244-kube-api-access-m6xsz\") pod \"console-57b6dbc6d6-qn8wb\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.549392 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.549376 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6n22\" (UniqueName: \"kubernetes.io/projected/84d86e1c-4a2f-4568-aaf1-d14b11521f53-kube-api-access-g6n22\") pod \"downloads-694c955b9-9xh9f\" (UID: \"84d86e1c-4a2f-4568-aaf1-d14b11521f53\") " pod="openshift-console/downloads-694c955b9-9xh9f" Jun 11 18:37:28.589248 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.589215 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 18:37:28.599279 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.599258 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 18:37:28.606622 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.606593 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" Jun 11 18:37:28.697006 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.696902 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-kqcj8" Jun 11 18:37:28.708428 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.706494 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:28.714107 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.713824 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-694c955b9-9xh9f" Jun 11 18:37:28.777035 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.776940 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j"] Jun 11 18:37:28.811203 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.810891 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx"] Jun 11 18:37:28.816000 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:37:28.815949 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7805e125_ba9b_4713_ac48_3678fba0895d.slice/crio-21fb3a322707058a70c0a9020b42fbb85b0210a065a09e717754ae3947db6a9b WatchSource:0}: Error finding container 21fb3a322707058a70c0a9020b42fbb85b0210a065a09e717754ae3947db6a9b: Status 404 returned error can't find the container with id 21fb3a322707058a70c0a9020b42fbb85b0210a065a09e717754ae3947db6a9b Jun 11 18:37:28.832656 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.832599 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8"] Jun 11 18:37:28.835481 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:37:28.835452 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21ff8dcc_639e_4865_af63_8f609ed3544e.slice/crio-d6e6db26624b1cc351403fb20c744c62395ab98e448728f50b9e7592013ea033 WatchSource:0}: Error finding container d6e6db26624b1cc351403fb20c744c62395ab98e448728f50b9e7592013ea033: Status 404 returned error can't find the container with id d6e6db26624b1cc351403fb20c744c62395ab98e448728f50b9e7592013ea033 Jun 11 18:37:28.878070 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.878034 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57b6dbc6d6-qn8wb"] Jun 11 18:37:28.884710 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:37:28.884678 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6096325_84a3_4c44_ba18_b4d74e5dd244.slice/crio-ad1ba33df783c5fd96ec50b8482c7ea8e4cb0f166331bcfa51905ea7df5b79c1 WatchSource:0}: Error finding container ad1ba33df783c5fd96ec50b8482c7ea8e4cb0f166331bcfa51905ea7df5b79c1: Status 404 returned error can't find the container with id ad1ba33df783c5fd96ec50b8482c7ea8e4cb0f166331bcfa51905ea7df5b79c1 Jun 11 18:37:28.895298 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.895249 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-kqcj8"] Jun 11 18:37:28.899500 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:37:28.899478 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f9034ca_6ad4_4c91_8103_55ebb839277f.slice/crio-a9eaa93693a3f593e71b17a2462e068537e820d6cec35bb0b35effe445c13831 WatchSource:0}: Error finding container a9eaa93693a3f593e71b17a2462e068537e820d6cec35bb0b35effe445c13831: Status 404 returned error can't find the container with id a9eaa93693a3f593e71b17a2462e068537e820d6cec35bb0b35effe445c13831 Jun 11 18:37:28.918855 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:28.918828 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-694c955b9-9xh9f"] Jun 11 18:37:28.921833 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:37:28.921806 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84d86e1c_4a2f_4568_aaf1_d14b11521f53.slice/crio-b1272036d9c0af8759bf18d5533b4fdb9572a2c0e5291cd9116c8d54cc79f815 WatchSource:0}: Error finding container b1272036d9c0af8759bf18d5533b4fdb9572a2c0e5291cd9116c8d54cc79f815: Status 404 returned error can't find the container with id b1272036d9c0af8759bf18d5533b4fdb9572a2c0e5291cd9116c8d54cc79f815 Jun 11 18:37:29.338655 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.338614 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f5b8b6fc8-97jhj" Jun 11 18:37:29.484522 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.484486 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-kqcj8" event={"ID":"3f9034ca-6ad4-4c91-8103-55ebb839277f","Type":"ContainerStarted","Data":"a9eaa93693a3f593e71b17a2462e068537e820d6cec35bb0b35effe445c13831"} Jun 11 18:37:29.486085 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.486052 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" event={"ID":"21ff8dcc-639e-4865-af63-8f609ed3544e","Type":"ContainerStarted","Data":"d6e6db26624b1cc351403fb20c744c62395ab98e448728f50b9e7592013ea033"} Jun 11 18:37:29.491875 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.491426 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-694c955b9-9xh9f" event={"ID":"84d86e1c-4a2f-4568-aaf1-d14b11521f53","Type":"ContainerStarted","Data":"b1272036d9c0af8759bf18d5533b4fdb9572a2c0e5291cd9116c8d54cc79f815"} Jun 11 18:37:29.494859 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.494780 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" event={"ID":"828f0479-53ba-4372-a7b6-108bc6b8355b","Type":"ContainerStarted","Data":"b4cc830e9d5e030426fe619c4d8e1f14fa9cb995a9757174304e43dfd320c32c"} Jun 11 18:37:29.508933 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.508522 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" event={"ID":"7805e125-ba9b-4713-ac48-3678fba0895d","Type":"ContainerStarted","Data":"21fb3a322707058a70c0a9020b42fbb85b0210a065a09e717754ae3947db6a9b"} Jun 11 18:37:29.513920 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.513871 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b6dbc6d6-qn8wb" event={"ID":"f6096325-84a3-4c44-ba18-b4d74e5dd244","Type":"ContainerStarted","Data":"ad1ba33df783c5fd96ec50b8482c7ea8e4cb0f166331bcfa51905ea7df5b79c1"} Jun 11 18:37:29.678574 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.677058 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b4d785fdb-2f5q5"] Jun 11 18:37:29.681322 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.680920 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.691242 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.691160 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b4d785fdb-2f5q5"] Jun 11 18:37:29.697966 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.696049 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 11 18:37:29.749602 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.749520 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-trusted-ca-bundle\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.750005 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.749885 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-console-config\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.750278 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.749976 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6b365bc-720a-42a1-9271-30b89b8222b1-console-serving-cert\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.750583 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.750388 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-oauth-serving-cert\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.750583 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.750426 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6b365bc-720a-42a1-9271-30b89b8222b1-console-oauth-config\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.750583 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.750466 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjhnk\" (UniqueName: \"kubernetes.io/projected/c6b365bc-720a-42a1-9271-30b89b8222b1-kube-api-access-jjhnk\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.750583 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.750494 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-service-ca\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.851506 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.851362 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-console-config\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.851506 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.851415 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6b365bc-720a-42a1-9271-30b89b8222b1-console-serving-cert\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.851506 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.851468 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-oauth-serving-cert\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.851506 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.851496 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6b365bc-720a-42a1-9271-30b89b8222b1-console-oauth-config\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.851827 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.851528 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjhnk\" (UniqueName: \"kubernetes.io/projected/c6b365bc-720a-42a1-9271-30b89b8222b1-kube-api-access-jjhnk\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.851827 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.851555 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-service-ca\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.851827 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.851631 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-trusted-ca-bundle\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.852873 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.852812 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-trusted-ca-bundle\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.853733 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.853514 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-console-config\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.854576 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.854550 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-oauth-serving-cert\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.855870 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.855791 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-service-ca\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.874017 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.873955 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6b365bc-720a-42a1-9271-30b89b8222b1-console-oauth-config\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.880646 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.880587 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6b365bc-720a-42a1-9271-30b89b8222b1-console-serving-cert\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:29.880775 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:29.880651 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjhnk\" (UniqueName: \"kubernetes.io/projected/c6b365bc-720a-42a1-9271-30b89b8222b1-kube-api-access-jjhnk\") pod \"console-b4d785fdb-2f5q5\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:30.002938 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:30.002568 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:30.233947 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:30.233815 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b4d785fdb-2f5q5"] Jun 11 18:37:30.246039 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:37:30.245793 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6b365bc_720a_42a1_9271_30b89b8222b1.slice/crio-25dbcfc150b04bdede79bc33dba185c3c06a4f5db7de8538aaeab27a62406874 WatchSource:0}: Error finding container 25dbcfc150b04bdede79bc33dba185c3c06a4f5db7de8538aaeab27a62406874: Status 404 returned error can't find the container with id 25dbcfc150b04bdede79bc33dba185c3c06a4f5db7de8538aaeab27a62406874 Jun 11 18:37:30.525710 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:30.525612 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b4d785fdb-2f5q5" event={"ID":"c6b365bc-720a-42a1-9271-30b89b8222b1","Type":"ContainerStarted","Data":"25dbcfc150b04bdede79bc33dba185c3c06a4f5db7de8538aaeab27a62406874"} Jun 11 18:37:38.381012 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.380976 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dk499" Jun 11 18:37:38.573494 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.573453 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-kqcj8" event={"ID":"3f9034ca-6ad4-4c91-8103-55ebb839277f","Type":"ContainerStarted","Data":"cd93ff0825699a04b34b974655d1886e800e1a5a1a658b9bf9b77e07df61c600"} Jun 11 18:37:38.573731 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.573708 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-kqcj8" Jun 11 18:37:38.575713 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.575679 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" event={"ID":"21ff8dcc-639e-4865-af63-8f609ed3544e","Type":"ContainerStarted","Data":"9f9028c54ea6d0b58add36f794339399d283a67fab7b89a86cc719cb4e012262"} Jun 11 18:37:38.579091 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.578871 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" event={"ID":"828f0479-53ba-4372-a7b6-108bc6b8355b","Type":"ContainerStarted","Data":"d40db652c5ae187c8b769d6421cde4074173a76ff00c8a75e55d7bb1d5d99118"} Jun 11 18:37:38.579239 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.579136 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 18:37:38.580210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.580187 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-kqcj8" Jun 11 18:37:38.581440 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.581418 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 18:37:38.585516 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.585494 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" event={"ID":"7805e125-ba9b-4713-ac48-3678fba0895d","Type":"ContainerStarted","Data":"d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f"} Jun 11 18:37:38.585659 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.585629 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" event={"ID":"7805e125-ba9b-4713-ac48-3678fba0895d","Type":"ContainerStarted","Data":"497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02"} Jun 11 18:37:38.585773 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.585661 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" event={"ID":"7805e125-ba9b-4713-ac48-3678fba0895d","Type":"ContainerStarted","Data":"a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec"} Jun 11 18:37:38.587344 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.587317 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b6dbc6d6-qn8wb" event={"ID":"f6096325-84a3-4c44-ba18-b4d74e5dd244","Type":"ContainerStarted","Data":"14fb34a8859c537f2d3f436988e543d5fc72e631d7739c9ac33fdf1ee9213fd7"} Jun 11 18:37:38.589269 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.589231 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b4d785fdb-2f5q5" event={"ID":"c6b365bc-720a-42a1-9271-30b89b8222b1","Type":"ContainerStarted","Data":"04ad65c74f88d4d2b7cee6794d8a927b270afd653dd0861bc874642b51a91cce"} Jun 11 18:37:38.589968 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.589934 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-kqcj8" podStartSLOduration=1.5059863629999999 podStartE2EDuration="10.589921684s" totalImagesPullingTime="9.083935321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:37:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:38.588669673 +0000 UTC m=+100.156743181" watchObservedRunningTime="2026-06-11 18:37:38.589921684 +0000 UTC m=+100.157995165" Jun 11 18:37:38.607457 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.607416 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" podStartSLOduration=1.357863925 podStartE2EDuration="10.607402665s" totalImagesPullingTime="9.24953874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:37:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:38.606200524 +0000 UTC m=+100.174273998" watchObservedRunningTime="2026-06-11 18:37:38.607402665 +0000 UTC m=+100.175476399" Jun 11 18:37:38.627695 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.627637 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" podStartSLOduration=1.385644774 podStartE2EDuration="10.627621046s" totalImagesPullingTime="9.241976272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:37:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:38.62636447 +0000 UTC m=+100.194437976" watchObservedRunningTime="2026-06-11 18:37:38.627621046 +0000 UTC m=+100.195694527" Jun 11 18:37:38.664040 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.663954 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57b6dbc6d6-qn8wb" podStartSLOduration=1.512003993 podStartE2EDuration="10.663933697s" totalImagesPullingTime="9.151929704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:37:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:38.662964993 +0000 UTC m=+100.231038469" watchObservedRunningTime="2026-06-11 18:37:38.663933697 +0000 UTC m=+100.232007177" Jun 11 18:37:38.678407 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.678351 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" podStartSLOduration=1.531124648 podStartE2EDuration="10.678333423s" totalImagesPullingTime="9.147208775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:37:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:38.676559687 +0000 UTC m=+100.244633168" watchObservedRunningTime="2026-06-11 18:37:38.678333423 +0000 UTC m=+100.246406904" Jun 11 18:37:38.695572 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.695519 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b4d785fdb-2f5q5" podStartSLOduration=1.905428629 podStartE2EDuration="9.695497745s" totalImagesPullingTime="7.790069116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:37:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:38.694466837 +0000 UTC m=+100.262540317" watchObservedRunningTime="2026-06-11 18:37:38.695497745 +0000 UTC m=+100.263571227" Jun 11 18:37:38.707043 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.707014 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:38.707268 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.707089 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:38.712727 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:38.712699 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:39.599199 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:39.599157 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:37:40.004049 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:40.003963 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:40.004049 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:40.004030 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:40.009990 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:40.009953 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:40.602656 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:40.602627 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:37:40.699015 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:40.698985 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57b6dbc6d6-qn8wb"] Jun 11 18:37:44.616697 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.614914 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5zhj9"] Jun 11 18:37:44.644919 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.644565 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.648606 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.647391 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 11 18:37:44.648606 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.647655 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qjvwn\"" Jun 11 18:37:44.648606 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.647970 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 11 18:37:44.648606 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.648247 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 11 18:37:44.648606 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.648428 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 11 18:37:44.825196 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.824622 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aad367a5-3066-4955-9163-578131a93d8e-node-exporter-textfile\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.825196 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.824715 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aad367a5-3066-4955-9163-578131a93d8e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.825196 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.824754 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aad367a5-3066-4955-9163-578131a93d8e-sys\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.825196 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.824816 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aad367a5-3066-4955-9163-578131a93d8e-node-exporter-tls\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.825196 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.824847 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aad367a5-3066-4955-9163-578131a93d8e-root\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.825196 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.824876 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aad367a5-3066-4955-9163-578131a93d8e-node-exporter-accelerators-collector-config\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.825196 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.824903 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aad367a5-3066-4955-9163-578131a93d8e-node-exporter-wtmp\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.825196 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.824935 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aad367a5-3066-4955-9163-578131a93d8e-metrics-client-ca\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.825196 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.824968 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsf8d\" (UniqueName: \"kubernetes.io/projected/aad367a5-3066-4955-9163-578131a93d8e-kube-api-access-bsf8d\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.925995 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.925909 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aad367a5-3066-4955-9163-578131a93d8e-metrics-client-ca\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.925995 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.925960 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsf8d\" (UniqueName: \"kubernetes.io/projected/aad367a5-3066-4955-9163-578131a93d8e-kube-api-access-bsf8d\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.925995 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.925986 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aad367a5-3066-4955-9163-578131a93d8e-node-exporter-textfile\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.926296 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.926048 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aad367a5-3066-4955-9163-578131a93d8e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.926296 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.926081 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aad367a5-3066-4955-9163-578131a93d8e-sys\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.926296 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.926119 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aad367a5-3066-4955-9163-578131a93d8e-node-exporter-tls\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.926296 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.926149 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aad367a5-3066-4955-9163-578131a93d8e-root\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.926296 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.926208 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aad367a5-3066-4955-9163-578131a93d8e-node-exporter-accelerators-collector-config\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.926296 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.926236 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aad367a5-3066-4955-9163-578131a93d8e-node-exporter-wtmp\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.927639 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.927562 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aad367a5-3066-4955-9163-578131a93d8e-node-exporter-textfile\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.927639 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.927637 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aad367a5-3066-4955-9163-578131a93d8e-root\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.927639 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.927639 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aad367a5-3066-4955-9163-578131a93d8e-sys\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.927893 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.927788 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aad367a5-3066-4955-9163-578131a93d8e-node-exporter-wtmp\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.928668 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.928167 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aad367a5-3066-4955-9163-578131a93d8e-metrics-client-ca\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.928668 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.928538 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aad367a5-3066-4955-9163-578131a93d8e-node-exporter-accelerators-collector-config\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.930917 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.930884 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aad367a5-3066-4955-9163-578131a93d8e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.931036 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.931013 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aad367a5-3066-4955-9163-578131a93d8e-node-exporter-tls\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.939108 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.939087 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsf8d\" (UniqueName: \"kubernetes.io/projected/aad367a5-3066-4955-9163-578131a93d8e-kube-api-access-bsf8d\") pod \"node-exporter-5zhj9\" (UID: \"aad367a5-3066-4955-9163-578131a93d8e\") " pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:44.960310 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:44.960278 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5zhj9" Jun 11 18:37:51.673896 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:37:51.673860 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaad367a5_3066_4955_9163_578131a93d8e.slice/crio-76c1006240f1e4f11136a0f976ed4dd1271cd79e48178af58aed5e37f6488220 WatchSource:0}: Error finding container 76c1006240f1e4f11136a0f976ed4dd1271cd79e48178af58aed5e37f6488220: Status 404 returned error can't find the container with id 76c1006240f1e4f11136a0f976ed4dd1271cd79e48178af58aed5e37f6488220 Jun 11 18:37:52.660883 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:52.660844 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5zhj9" event={"ID":"aad367a5-3066-4955-9163-578131a93d8e","Type":"ContainerStarted","Data":"76c1006240f1e4f11136a0f976ed4dd1271cd79e48178af58aed5e37f6488220"} Jun 11 18:37:52.662840 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:52.662807 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-694c955b9-9xh9f" event={"ID":"84d86e1c-4a2f-4568-aaf1-d14b11521f53","Type":"ContainerStarted","Data":"f9ef44d73abfe0d6634a3c5d4fb20a86f76f0c053aa3f89f46624cd51efb4b9f"} Jun 11 18:37:52.663059 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:52.663035 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-694c955b9-9xh9f" Jun 11 18:37:52.673038 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:52.673014 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-694c955b9-9xh9f" Jun 11 18:37:52.684319 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:52.684263 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-694c955b9-9xh9f" podStartSLOduration=1.681104836 podStartE2EDuration="24.684245359s" totalImagesPullingTime="23.003140523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:37:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:52.681432793 +0000 UTC m=+114.249506276" watchObservedRunningTime="2026-06-11 18:37:52.684245359 +0000 UTC m=+114.252318840" Jun 11 18:37:53.669742 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:53.669700 2561 generic.go:356] "Generic (PLEG): container finished" podID="aad367a5-3066-4955-9163-578131a93d8e" containerID="7f31d4b80543822dcf958f04f3a5130f1503045c68d9ce307c8f1d4120f1d1d7" exitCode=0 Jun 11 18:37:53.670001 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:53.669785 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5zhj9" event={"ID":"aad367a5-3066-4955-9163-578131a93d8e","Type":"ContainerDied","Data":"7f31d4b80543822dcf958f04f3a5130f1503045c68d9ce307c8f1d4120f1d1d7"} Jun 11 18:37:54.677125 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:54.677083 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5zhj9" event={"ID":"aad367a5-3066-4955-9163-578131a93d8e","Type":"ContainerStarted","Data":"d017e38949d738570fbadf80bb7ce6cfa161b2eba2452cd8998996f08673c8be"} Jun 11 18:37:54.677595 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:54.677157 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5zhj9" event={"ID":"aad367a5-3066-4955-9163-578131a93d8e","Type":"ContainerStarted","Data":"cd17530542a9568ae33951b6c11a730d3ac28a7e041a66ac33b20cf76219265d"} Jun 11 18:37:54.697838 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:54.697779 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5zhj9" podStartSLOduration=9.700076789 podStartE2EDuration="10.697758685s" totalImagesPullingTime="910.139301ms" totalInitContainerRuntime="87.542595ms" isStatefulPod=true podCreationTimestamp="2026-06-11 18:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:37:54.697082726 +0000 UTC m=+116.265156207" watchObservedRunningTime="2026-06-11 18:37:54.697758685 +0000 UTC m=+116.265832164" Jun 11 18:37:56.688254 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:56.688214 2561 generic.go:356] "Generic (PLEG): container finished" podID="6df2530a-816a-4737-8815-4c537d8a61c4" containerID="42377da449cad637e6c69a42bcdb6050b8cde2ef4e63519e29dc8927724a231b" exitCode=0 Jun 11 18:37:56.688859 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:56.688298 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" event={"ID":"6df2530a-816a-4737-8815-4c537d8a61c4","Type":"ContainerDied","Data":"42377da449cad637e6c69a42bcdb6050b8cde2ef4e63519e29dc8927724a231b"} Jun 11 18:37:56.688859 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:56.688729 2561 scope.go:117] "RemoveContainer" containerID="42377da449cad637e6c69a42bcdb6050b8cde2ef4e63519e29dc8927724a231b" Jun 11 18:37:57.696737 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:57.696694 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7c578b78fc-6kgqh" event={"ID":"6df2530a-816a-4737-8815-4c537d8a61c4","Type":"ContainerStarted","Data":"e6a53f85dcbd32492d890c444c76801ac89a78d14e5e36674ad5ea22d67c46a3"} Jun 11 18:37:57.698688 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:57.698658 2561 generic.go:356] "Generic (PLEG): container finished" podID="64cd27df-ed04-40a9-89c5-0df92402e31b" containerID="26acb70c5d7fd7eb0470fe6ef3936c06ee7d1962f0dd3a46c31f2ef13c3f6bf5" exitCode=0 Jun 11 18:37:57.698835 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:57.698722 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" event={"ID":"64cd27df-ed04-40a9-89c5-0df92402e31b","Type":"ContainerDied","Data":"26acb70c5d7fd7eb0470fe6ef3936c06ee7d1962f0dd3a46c31f2ef13c3f6bf5"} Jun 11 18:37:57.699077 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:57.699059 2561 scope.go:117] "RemoveContainer" containerID="26acb70c5d7fd7eb0470fe6ef3936c06ee7d1962f0dd3a46c31f2ef13c3f6bf5" Jun 11 18:37:58.706776 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:37:58.706729 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-h7sf5" event={"ID":"64cd27df-ed04-40a9-89c5-0df92402e31b","Type":"ContainerStarted","Data":"d5d5a625694a683a8d975ae7194d79c9d029cc7ec2d2335cadf86800362faa4e"} Jun 11 18:38:07.633973 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:07.633921 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-57b6dbc6d6-qn8wb" podUID="f6096325-84a3-4c44-ba18-b4d74e5dd244" containerName="console" containerID="cri-o://14fb34a8859c537f2d3f436988e543d5fc72e631d7739c9ac33fdf1ee9213fd7" gracePeriod=15 Jun 11 18:38:07.750906 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:07.750873 2561 generic.go:356] "Generic (PLEG): container finished" podID="f6096325-84a3-4c44-ba18-b4d74e5dd244" containerID="14fb34a8859c537f2d3f436988e543d5fc72e631d7739c9ac33fdf1ee9213fd7" exitCode=0 Jun 11 18:38:07.751047 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:07.750948 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b6dbc6d6-qn8wb" event={"ID":"f6096325-84a3-4c44-ba18-b4d74e5dd244","Type":"ContainerDied","Data":"14fb34a8859c537f2d3f436988e543d5fc72e631d7739c9ac33fdf1ee9213fd7"} Jun 11 18:38:07.921191 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:07.921156 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:38:08.044576 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.044534 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-oauth-config\") pod \"f6096325-84a3-4c44-ba18-b4d74e5dd244\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " Jun 11 18:38:08.044576 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.044577 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-oauth-serving-cert\") pod \"f6096325-84a3-4c44-ba18-b4d74e5dd244\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " Jun 11 18:38:08.044828 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.044611 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6xsz\" (UniqueName: \"kubernetes.io/projected/f6096325-84a3-4c44-ba18-b4d74e5dd244-kube-api-access-m6xsz\") pod \"f6096325-84a3-4c44-ba18-b4d74e5dd244\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " Jun 11 18:38:08.044828 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.044632 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-config\") pod \"f6096325-84a3-4c44-ba18-b4d74e5dd244\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " Jun 11 18:38:08.044828 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.044654 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-service-ca\") pod \"f6096325-84a3-4c44-ba18-b4d74e5dd244\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " Jun 11 18:38:08.044828 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.044674 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-serving-cert\") pod \"f6096325-84a3-4c44-ba18-b4d74e5dd244\" (UID: \"f6096325-84a3-4c44-ba18-b4d74e5dd244\") " Jun 11 18:38:08.045040 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.045005 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-config" (OuterVolumeSpecName: "console-config") pod "f6096325-84a3-4c44-ba18-b4d74e5dd244" (UID: "f6096325-84a3-4c44-ba18-b4d74e5dd244"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 18:38:08.045096 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.045011 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-service-ca" (OuterVolumeSpecName: "service-ca") pod "f6096325-84a3-4c44-ba18-b4d74e5dd244" (UID: "f6096325-84a3-4c44-ba18-b4d74e5dd244"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 18:38:08.045096 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.045017 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f6096325-84a3-4c44-ba18-b4d74e5dd244" (UID: "f6096325-84a3-4c44-ba18-b4d74e5dd244"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 18:38:08.047012 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.046981 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f6096325-84a3-4c44-ba18-b4d74e5dd244" (UID: "f6096325-84a3-4c44-ba18-b4d74e5dd244"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 18:38:08.047012 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.046997 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f6096325-84a3-4c44-ba18-b4d74e5dd244" (UID: "f6096325-84a3-4c44-ba18-b4d74e5dd244"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 18:38:08.047136 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.046992 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6096325-84a3-4c44-ba18-b4d74e5dd244-kube-api-access-m6xsz" (OuterVolumeSpecName: "kube-api-access-m6xsz") pod "f6096325-84a3-4c44-ba18-b4d74e5dd244" (UID: "f6096325-84a3-4c44-ba18-b4d74e5dd244"). InnerVolumeSpecName "kube-api-access-m6xsz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 18:38:08.145864 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.145783 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-oauth-config\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:38:08.145864 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.145813 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-oauth-serving-cert\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:38:08.145864 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.145823 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6xsz\" (UniqueName: \"kubernetes.io/projected/f6096325-84a3-4c44-ba18-b4d74e5dd244-kube-api-access-m6xsz\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:38:08.145864 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.145834 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-config\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:38:08.145864 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.145842 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6096325-84a3-4c44-ba18-b4d74e5dd244-service-ca\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:38:08.145864 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.145851 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6096325-84a3-4c44-ba18-b4d74e5dd244-console-serving-cert\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:38:08.756482 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.756451 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b6dbc6d6-qn8wb" Jun 11 18:38:08.756913 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.756453 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b6dbc6d6-qn8wb" event={"ID":"f6096325-84a3-4c44-ba18-b4d74e5dd244","Type":"ContainerDied","Data":"ad1ba33df783c5fd96ec50b8482c7ea8e4cb0f166331bcfa51905ea7df5b79c1"} Jun 11 18:38:08.756913 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.756596 2561 scope.go:117] "RemoveContainer" containerID="14fb34a8859c537f2d3f436988e543d5fc72e631d7739c9ac33fdf1ee9213fd7" Jun 11 18:38:08.779803 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.779778 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57b6dbc6d6-qn8wb"] Jun 11 18:38:08.786674 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.786652 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57b6dbc6d6-qn8wb"] Jun 11 18:38:08.919595 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.919563 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6096325-84a3-4c44-ba18-b4d74e5dd244" path="/var/lib/kubelet/pods/f6096325-84a3-4c44-ba18-b4d74e5dd244/volumes" Jun 11 18:38:08.927753 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:08.927722 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b4d785fdb-2f5q5"] Jun 11 18:38:11.772880 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:11.772845 2561 generic.go:356] "Generic (PLEG): container finished" podID="8e0dc434-3567-48d2-980c-1747934625aa" containerID="896f749ca3f15fc58b992078abaf1854edb31af223676eb4efe03c518f4030a9" exitCode=0 Jun 11 18:38:11.773344 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:11.772918 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" event={"ID":"8e0dc434-3567-48d2-980c-1747934625aa","Type":"ContainerDied","Data":"896f749ca3f15fc58b992078abaf1854edb31af223676eb4efe03c518f4030a9"} Jun 11 18:38:11.773344 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:11.773298 2561 scope.go:117] "RemoveContainer" containerID="896f749ca3f15fc58b992078abaf1854edb31af223676eb4efe03c518f4030a9" Jun 11 18:38:12.779386 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:12.779353 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7ccbfd59dc-b96qg" event={"ID":"8e0dc434-3567-48d2-980c-1747934625aa","Type":"ContainerStarted","Data":"0b435e97cae28ec68175ae884ebe0103af3164cfe7dbe8355bc8e3d95a265d4f"} Jun 11 18:38:33.948532 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:33.948487 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-b4d785fdb-2f5q5" podUID="c6b365bc-720a-42a1-9271-30b89b8222b1" containerName="console" containerID="cri-o://04ad65c74f88d4d2b7cee6794d8a927b270afd653dd0861bc874642b51a91cce" gracePeriod=15 Jun 11 18:38:34.234304 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.234272 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:38:34.275356 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.275322 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-service-ca\") pod \"c6b365bc-720a-42a1-9271-30b89b8222b1\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " Jun 11 18:38:34.275528 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.275362 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-console-config\") pod \"c6b365bc-720a-42a1-9271-30b89b8222b1\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " Jun 11 18:38:34.275528 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.275398 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6b365bc-720a-42a1-9271-30b89b8222b1-console-serving-cert\") pod \"c6b365bc-720a-42a1-9271-30b89b8222b1\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " Jun 11 18:38:34.275528 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.275426 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6b365bc-720a-42a1-9271-30b89b8222b1-console-oauth-config\") pod \"c6b365bc-720a-42a1-9271-30b89b8222b1\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " Jun 11 18:38:34.275528 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.275453 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-trusted-ca-bundle\") pod \"c6b365bc-720a-42a1-9271-30b89b8222b1\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " Jun 11 18:38:34.275774 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.275648 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjhnk\" (UniqueName: \"kubernetes.io/projected/c6b365bc-720a-42a1-9271-30b89b8222b1-kube-api-access-jjhnk\") pod \"c6b365bc-720a-42a1-9271-30b89b8222b1\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " Jun 11 18:38:34.275774 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.275702 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-oauth-serving-cert\") pod \"c6b365bc-720a-42a1-9271-30b89b8222b1\" (UID: \"c6b365bc-720a-42a1-9271-30b89b8222b1\") " Jun 11 18:38:34.275878 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.275770 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-service-ca" (OuterVolumeSpecName: "service-ca") pod "c6b365bc-720a-42a1-9271-30b89b8222b1" (UID: "c6b365bc-720a-42a1-9271-30b89b8222b1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 18:38:34.275878 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.275820 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-console-config" (OuterVolumeSpecName: "console-config") pod "c6b365bc-720a-42a1-9271-30b89b8222b1" (UID: "c6b365bc-720a-42a1-9271-30b89b8222b1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 18:38:34.275977 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.275892 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c6b365bc-720a-42a1-9271-30b89b8222b1" (UID: "c6b365bc-720a-42a1-9271-30b89b8222b1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 18:38:34.276188 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.276145 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c6b365bc-720a-42a1-9271-30b89b8222b1" (UID: "c6b365bc-720a-42a1-9271-30b89b8222b1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 18:38:34.276276 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.276154 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-service-ca\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:38:34.276276 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.276225 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-console-config\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:38:34.276276 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.276239 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-trusted-ca-bundle\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:38:34.277862 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.277830 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6b365bc-720a-42a1-9271-30b89b8222b1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c6b365bc-720a-42a1-9271-30b89b8222b1" (UID: "c6b365bc-720a-42a1-9271-30b89b8222b1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 18:38:34.277862 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.277839 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6b365bc-720a-42a1-9271-30b89b8222b1-kube-api-access-jjhnk" (OuterVolumeSpecName: "kube-api-access-jjhnk") pod "c6b365bc-720a-42a1-9271-30b89b8222b1" (UID: "c6b365bc-720a-42a1-9271-30b89b8222b1"). InnerVolumeSpecName "kube-api-access-jjhnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 18:38:34.277992 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.277872 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6b365bc-720a-42a1-9271-30b89b8222b1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c6b365bc-720a-42a1-9271-30b89b8222b1" (UID: "c6b365bc-720a-42a1-9271-30b89b8222b1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 18:38:34.377154 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.377116 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjhnk\" (UniqueName: \"kubernetes.io/projected/c6b365bc-720a-42a1-9271-30b89b8222b1-kube-api-access-jjhnk\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:38:34.377154 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.377148 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6b365bc-720a-42a1-9271-30b89b8222b1-oauth-serving-cert\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:38:34.377154 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.377160 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6b365bc-720a-42a1-9271-30b89b8222b1-console-serving-cert\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:38:34.377423 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.377197 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6b365bc-720a-42a1-9271-30b89b8222b1-console-oauth-config\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:38:34.877676 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.877637 2561 generic.go:356] "Generic (PLEG): container finished" podID="c6b365bc-720a-42a1-9271-30b89b8222b1" containerID="04ad65c74f88d4d2b7cee6794d8a927b270afd653dd0861bc874642b51a91cce" exitCode=0 Jun 11 18:38:34.877844 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.877686 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b4d785fdb-2f5q5" Jun 11 18:38:34.877844 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.877716 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b4d785fdb-2f5q5" event={"ID":"c6b365bc-720a-42a1-9271-30b89b8222b1","Type":"ContainerDied","Data":"04ad65c74f88d4d2b7cee6794d8a927b270afd653dd0861bc874642b51a91cce"} Jun 11 18:38:34.877844 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.877767 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b4d785fdb-2f5q5" event={"ID":"c6b365bc-720a-42a1-9271-30b89b8222b1","Type":"ContainerDied","Data":"25dbcfc150b04bdede79bc33dba185c3c06a4f5db7de8538aaeab27a62406874"} Jun 11 18:38:34.877844 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.877788 2561 scope.go:117] "RemoveContainer" containerID="04ad65c74f88d4d2b7cee6794d8a927b270afd653dd0861bc874642b51a91cce" Jun 11 18:38:34.886440 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.886426 2561 scope.go:117] "RemoveContainer" containerID="04ad65c74f88d4d2b7cee6794d8a927b270afd653dd0861bc874642b51a91cce" Jun 11 18:38:34.886745 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:38:34.886728 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04ad65c74f88d4d2b7cee6794d8a927b270afd653dd0861bc874642b51a91cce\": container with ID starting with 04ad65c74f88d4d2b7cee6794d8a927b270afd653dd0861bc874642b51a91cce not found: ID does not exist" containerID="04ad65c74f88d4d2b7cee6794d8a927b270afd653dd0861bc874642b51a91cce" Jun 11 18:38:34.886807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.886761 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ad65c74f88d4d2b7cee6794d8a927b270afd653dd0861bc874642b51a91cce"} err="failed to get container status \"04ad65c74f88d4d2b7cee6794d8a927b270afd653dd0861bc874642b51a91cce\": rpc error: code = NotFound desc = could not find container \"04ad65c74f88d4d2b7cee6794d8a927b270afd653dd0861bc874642b51a91cce\": container with ID starting with 04ad65c74f88d4d2b7cee6794d8a927b270afd653dd0861bc874642b51a91cce not found: ID does not exist" Jun 11 18:38:34.898726 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.898702 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b4d785fdb-2f5q5"] Jun 11 18:38:34.902673 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.902651 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b4d785fdb-2f5q5"] Jun 11 18:38:34.917373 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:38:34.917345 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6b365bc-720a-42a1-9271-30b89b8222b1" path="/var/lib/kubelet/pods/c6b365bc-720a-42a1-9271-30b89b8222b1/volumes" Jun 11 18:40:58.857054 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:40:58.857018 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 18:40:58.859218 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:40:58.859194 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 18:40:58.867084 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:40:58.867066 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 18:40:58.868821 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:40:58.868802 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 18:40:58.870791 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:40:58.870773 2561 kubelet.go:1648] "Image garbage collection succeeded" Jun 11 18:41:22.102723 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.102692 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr"] Jun 11 18:41:22.105028 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.103006 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6096325-84a3-4c44-ba18-b4d74e5dd244" containerName="console" Jun 11 18:41:22.105028 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.103017 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6096325-84a3-4c44-ba18-b4d74e5dd244" containerName="console" Jun 11 18:41:22.105028 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.103030 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6b365bc-720a-42a1-9271-30b89b8222b1" containerName="console" Jun 11 18:41:22.105028 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.103035 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6b365bc-720a-42a1-9271-30b89b8222b1" containerName="console" Jun 11 18:41:22.105028 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.103098 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6b365bc-720a-42a1-9271-30b89b8222b1" containerName="console" Jun 11 18:41:22.105028 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.103107 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6096325-84a3-4c44-ba18-b4d74e5dd244" containerName="console" Jun 11 18:41:22.106022 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.106006 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.109485 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.109444 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 11 18:41:22.109485 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.109453 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 11 18:41:22.110034 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.110013 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 11 18:41:22.110228 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.110063 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-d7tvg\"" Jun 11 18:41:22.110339 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.110324 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 11 18:41:22.128502 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.128469 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr"] Jun 11 18:41:22.172109 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.172074 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6a2590cc-b669-4c75-aa43-caf07a2615d1-manifests\") pod \"opendatahub-operator-controller-manager-54d5cbc44f-vr2sr\" (UID: \"6a2590cc-b669-4c75-aa43-caf07a2615d1\") " pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.172109 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.172121 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhmjr\" (UniqueName: \"kubernetes.io/projected/6a2590cc-b669-4c75-aa43-caf07a2615d1-kube-api-access-zhmjr\") pod \"opendatahub-operator-controller-manager-54d5cbc44f-vr2sr\" (UID: \"6a2590cc-b669-4c75-aa43-caf07a2615d1\") " pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.172381 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.172145 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a2590cc-b669-4c75-aa43-caf07a2615d1-tmp\") pod \"opendatahub-operator-controller-manager-54d5cbc44f-vr2sr\" (UID: \"6a2590cc-b669-4c75-aa43-caf07a2615d1\") " pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.172381 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.172216 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a2590cc-b669-4c75-aa43-caf07a2615d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-54d5cbc44f-vr2sr\" (UID: \"6a2590cc-b669-4c75-aa43-caf07a2615d1\") " pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.172381 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.172256 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a2590cc-b669-4c75-aa43-caf07a2615d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54d5cbc44f-vr2sr\" (UID: \"6a2590cc-b669-4c75-aa43-caf07a2615d1\") " pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.273032 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.272995 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6a2590cc-b669-4c75-aa43-caf07a2615d1-manifests\") pod \"opendatahub-operator-controller-manager-54d5cbc44f-vr2sr\" (UID: \"6a2590cc-b669-4c75-aa43-caf07a2615d1\") " pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.273032 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.273036 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhmjr\" (UniqueName: \"kubernetes.io/projected/6a2590cc-b669-4c75-aa43-caf07a2615d1-kube-api-access-zhmjr\") pod \"opendatahub-operator-controller-manager-54d5cbc44f-vr2sr\" (UID: \"6a2590cc-b669-4c75-aa43-caf07a2615d1\") " pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.273312 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.273060 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a2590cc-b669-4c75-aa43-caf07a2615d1-tmp\") pod \"opendatahub-operator-controller-manager-54d5cbc44f-vr2sr\" (UID: \"6a2590cc-b669-4c75-aa43-caf07a2615d1\") " pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.273312 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.273104 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a2590cc-b669-4c75-aa43-caf07a2615d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-54d5cbc44f-vr2sr\" (UID: \"6a2590cc-b669-4c75-aa43-caf07a2615d1\") " pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.273312 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.273135 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a2590cc-b669-4c75-aa43-caf07a2615d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54d5cbc44f-vr2sr\" (UID: \"6a2590cc-b669-4c75-aa43-caf07a2615d1\") " pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.273504 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.273468 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6a2590cc-b669-4c75-aa43-caf07a2615d1-manifests\") pod \"opendatahub-operator-controller-manager-54d5cbc44f-vr2sr\" (UID: \"6a2590cc-b669-4c75-aa43-caf07a2615d1\") " pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.275408 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.275380 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a2590cc-b669-4c75-aa43-caf07a2615d1-tmp\") pod \"opendatahub-operator-controller-manager-54d5cbc44f-vr2sr\" (UID: \"6a2590cc-b669-4c75-aa43-caf07a2615d1\") " pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.275642 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.275624 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a2590cc-b669-4c75-aa43-caf07a2615d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-54d5cbc44f-vr2sr\" (UID: \"6a2590cc-b669-4c75-aa43-caf07a2615d1\") " pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.275716 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.275663 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a2590cc-b669-4c75-aa43-caf07a2615d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54d5cbc44f-vr2sr\" (UID: \"6a2590cc-b669-4c75-aa43-caf07a2615d1\") " pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.284814 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.284790 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhmjr\" (UniqueName: \"kubernetes.io/projected/6a2590cc-b669-4c75-aa43-caf07a2615d1-kube-api-access-zhmjr\") pod \"opendatahub-operator-controller-manager-54d5cbc44f-vr2sr\" (UID: \"6a2590cc-b669-4c75-aa43-caf07a2615d1\") " pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.417161 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.417079 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:22.556035 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.555959 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr"] Jun 11 18:41:22.558760 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:41:22.558735 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a2590cc_b669_4c75_aa43_caf07a2615d1.slice/crio-e84126bec9162473bc32f66e13fc0cf321de4e42949037dc1ddce0b73a759bec WatchSource:0}: Error finding container e84126bec9162473bc32f66e13fc0cf321de4e42949037dc1ddce0b73a759bec: Status 404 returned error can't find the container with id e84126bec9162473bc32f66e13fc0cf321de4e42949037dc1ddce0b73a759bec Jun 11 18:41:22.560515 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:22.560499 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 18:41:23.558225 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:23.558165 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" event={"ID":"6a2590cc-b669-4c75-aa43-caf07a2615d1","Type":"ContainerStarted","Data":"e84126bec9162473bc32f66e13fc0cf321de4e42949037dc1ddce0b73a759bec"} Jun 11 18:41:25.576808 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:25.576713 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" event={"ID":"6a2590cc-b669-4c75-aa43-caf07a2615d1","Type":"ContainerStarted","Data":"980ca6c668024e4de8cbb5d42fa166bf2367ff692ec6004c32bf49af3deed78f"} Jun 11 18:41:26.582622 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:26.582586 2561 generic.go:356] "Generic (PLEG): container finished" podID="6a2590cc-b669-4c75-aa43-caf07a2615d1" containerID="980ca6c668024e4de8cbb5d42fa166bf2367ff692ec6004c32bf49af3deed78f" exitCode=0 Jun 11 18:41:26.583022 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:26.582663 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" event={"ID":"6a2590cc-b669-4c75-aa43-caf07a2615d1","Type":"ContainerDied","Data":"980ca6c668024e4de8cbb5d42fa166bf2367ff692ec6004c32bf49af3deed78f"} Jun 11 18:41:27.588002 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:27.587961 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" event={"ID":"6a2590cc-b669-4c75-aa43-caf07a2615d1","Type":"ContainerStarted","Data":"e4fda8862641762e0b02287891b6e6ad70eac6aff3aad0442c9d524fd66512a3"} Jun 11 18:41:27.588432 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:27.588067 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:27.608975 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:27.608920 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" podStartSLOduration=1.500385927 podStartE2EDuration="5.60890337s" totalImagesPullingTime="3.182212916s" totalInitContainerRuntime="926.304527ms" isStatefulPod=false podCreationTimestamp="2026-06-11 18:41:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:41:27.60631539 +0000 UTC m=+329.174388872" watchObservedRunningTime="2026-06-11 18:41:27.60890337 +0000 UTC m=+329.176976850" Jun 11 18:41:38.594486 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:38.594455 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-54d5cbc44f-vr2sr" Jun 11 18:41:41.276887 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.276852 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8d878499b-92tdm"] Jun 11 18:41:41.289245 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.289216 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8d878499b-92tdm" Jun 11 18:41:41.292018 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.291997 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 11 18:41:41.292167 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.292146 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qs28d\"" Jun 11 18:41:41.292855 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.292838 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 11 18:41:41.303933 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.303908 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8d878499b-92tdm"] Jun 11 18:41:41.343663 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.343629 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9fw9\" (UniqueName: \"kubernetes.io/projected/970e8ea0-a08b-46ea-badb-f9d914f344a1-kube-api-access-v9fw9\") pod \"kube-auth-proxy-8d878499b-92tdm\" (UID: \"970e8ea0-a08b-46ea-badb-f9d914f344a1\") " pod="openshift-ingress/kube-auth-proxy-8d878499b-92tdm" Jun 11 18:41:41.343843 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.343686 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/970e8ea0-a08b-46ea-badb-f9d914f344a1-tls-certs\") pod \"kube-auth-proxy-8d878499b-92tdm\" (UID: \"970e8ea0-a08b-46ea-badb-f9d914f344a1\") " pod="openshift-ingress/kube-auth-proxy-8d878499b-92tdm" Jun 11 18:41:41.343843 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.343757 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/970e8ea0-a08b-46ea-badb-f9d914f344a1-tmp\") pod \"kube-auth-proxy-8d878499b-92tdm\" (UID: \"970e8ea0-a08b-46ea-badb-f9d914f344a1\") " pod="openshift-ingress/kube-auth-proxy-8d878499b-92tdm" Jun 11 18:41:41.444271 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.444234 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9fw9\" (UniqueName: \"kubernetes.io/projected/970e8ea0-a08b-46ea-badb-f9d914f344a1-kube-api-access-v9fw9\") pod \"kube-auth-proxy-8d878499b-92tdm\" (UID: \"970e8ea0-a08b-46ea-badb-f9d914f344a1\") " pod="openshift-ingress/kube-auth-proxy-8d878499b-92tdm" Jun 11 18:41:41.444469 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.444298 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/970e8ea0-a08b-46ea-badb-f9d914f344a1-tls-certs\") pod \"kube-auth-proxy-8d878499b-92tdm\" (UID: \"970e8ea0-a08b-46ea-badb-f9d914f344a1\") " pod="openshift-ingress/kube-auth-proxy-8d878499b-92tdm" Jun 11 18:41:41.444469 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.444353 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/970e8ea0-a08b-46ea-badb-f9d914f344a1-tmp\") pod \"kube-auth-proxy-8d878499b-92tdm\" (UID: \"970e8ea0-a08b-46ea-badb-f9d914f344a1\") " pod="openshift-ingress/kube-auth-proxy-8d878499b-92tdm" Jun 11 18:41:41.446555 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.446532 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/970e8ea0-a08b-46ea-badb-f9d914f344a1-tmp\") pod \"kube-auth-proxy-8d878499b-92tdm\" (UID: \"970e8ea0-a08b-46ea-badb-f9d914f344a1\") " pod="openshift-ingress/kube-auth-proxy-8d878499b-92tdm" Jun 11 18:41:41.446888 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.446868 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/970e8ea0-a08b-46ea-badb-f9d914f344a1-tls-certs\") pod \"kube-auth-proxy-8d878499b-92tdm\" (UID: \"970e8ea0-a08b-46ea-badb-f9d914f344a1\") " pod="openshift-ingress/kube-auth-proxy-8d878499b-92tdm" Jun 11 18:41:41.477696 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.477664 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9fw9\" (UniqueName: \"kubernetes.io/projected/970e8ea0-a08b-46ea-badb-f9d914f344a1-kube-api-access-v9fw9\") pod \"kube-auth-proxy-8d878499b-92tdm\" (UID: \"970e8ea0-a08b-46ea-badb-f9d914f344a1\") " pod="openshift-ingress/kube-auth-proxy-8d878499b-92tdm" Jun 11 18:41:41.598559 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.598525 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8d878499b-92tdm" Jun 11 18:41:41.762775 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:41.762624 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8d878499b-92tdm"] Jun 11 18:41:41.764996 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:41:41.764962 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod970e8ea0_a08b_46ea_badb_f9d914f344a1.slice/crio-cb978945138b76875d92c44cbd238d14409d1a4791e1a1fec984c3155c14c2f8 WatchSource:0}: Error finding container cb978945138b76875d92c44cbd238d14409d1a4791e1a1fec984c3155c14c2f8: Status 404 returned error can't find the container with id cb978945138b76875d92c44cbd238d14409d1a4791e1a1fec984c3155c14c2f8 Jun 11 18:41:42.295790 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:42.295755 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-scn5g"] Jun 11 18:41:42.300331 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:42.300313 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-scn5g" Jun 11 18:41:42.302552 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:42.302531 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-h96jf\"" Jun 11 18:41:42.302793 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:42.302774 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 11 18:41:42.311648 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:42.311629 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-scn5g"] Jun 11 18:41:42.455419 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:42.455332 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6frn\" (UniqueName: \"kubernetes.io/projected/2744b8dd-170c-433e-b137-c5bdbc2ab252-kube-api-access-v6frn\") pod \"model-serving-api-5dd488fc48-scn5g\" (UID: \"2744b8dd-170c-433e-b137-c5bdbc2ab252\") " pod="opendatahub/model-serving-api-5dd488fc48-scn5g" Jun 11 18:41:42.455675 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:42.455442 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2744b8dd-170c-433e-b137-c5bdbc2ab252-tls-certs\") pod \"model-serving-api-5dd488fc48-scn5g\" (UID: \"2744b8dd-170c-433e-b137-c5bdbc2ab252\") " pod="opendatahub/model-serving-api-5dd488fc48-scn5g" Jun 11 18:41:42.556946 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:42.556853 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6frn\" (UniqueName: \"kubernetes.io/projected/2744b8dd-170c-433e-b137-c5bdbc2ab252-kube-api-access-v6frn\") pod \"model-serving-api-5dd488fc48-scn5g\" (UID: \"2744b8dd-170c-433e-b137-c5bdbc2ab252\") " pod="opendatahub/model-serving-api-5dd488fc48-scn5g" Jun 11 18:41:42.556946 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:42.556917 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2744b8dd-170c-433e-b137-c5bdbc2ab252-tls-certs\") pod \"model-serving-api-5dd488fc48-scn5g\" (UID: \"2744b8dd-170c-433e-b137-c5bdbc2ab252\") " pod="opendatahub/model-serving-api-5dd488fc48-scn5g" Jun 11 18:41:42.557194 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:41:42.557003 2561 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jun 11 18:41:42.557194 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:41:42.557054 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2744b8dd-170c-433e-b137-c5bdbc2ab252-tls-certs podName:2744b8dd-170c-433e-b137-c5bdbc2ab252 nodeName:}" failed. No retries permitted until 2026-06-11 18:41:43.057038586 +0000 UTC m=+344.625112045 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2744b8dd-170c-433e-b137-c5bdbc2ab252-tls-certs") pod "model-serving-api-5dd488fc48-scn5g" (UID: "2744b8dd-170c-433e-b137-c5bdbc2ab252") : secret "model-serving-api-tls" not found Jun 11 18:41:42.566231 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:42.566164 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6frn\" (UniqueName: \"kubernetes.io/projected/2744b8dd-170c-433e-b137-c5bdbc2ab252-kube-api-access-v6frn\") pod \"model-serving-api-5dd488fc48-scn5g\" (UID: \"2744b8dd-170c-433e-b137-c5bdbc2ab252\") " pod="opendatahub/model-serving-api-5dd488fc48-scn5g" Jun 11 18:41:42.651994 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:42.651950 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8d878499b-92tdm" event={"ID":"970e8ea0-a08b-46ea-badb-f9d914f344a1","Type":"ContainerStarted","Data":"cb978945138b76875d92c44cbd238d14409d1a4791e1a1fec984c3155c14c2f8"} Jun 11 18:41:43.060883 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:43.060841 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2744b8dd-170c-433e-b137-c5bdbc2ab252-tls-certs\") pod \"model-serving-api-5dd488fc48-scn5g\" (UID: \"2744b8dd-170c-433e-b137-c5bdbc2ab252\") " pod="opendatahub/model-serving-api-5dd488fc48-scn5g" Jun 11 18:41:43.063965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:43.063906 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2744b8dd-170c-433e-b137-c5bdbc2ab252-tls-certs\") pod \"model-serving-api-5dd488fc48-scn5g\" (UID: \"2744b8dd-170c-433e-b137-c5bdbc2ab252\") " pod="opendatahub/model-serving-api-5dd488fc48-scn5g" Jun 11 18:41:43.211325 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:43.211285 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-scn5g" Jun 11 18:41:43.380248 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:43.380218 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-scn5g"] Jun 11 18:41:43.383228 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:41:43.383191 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2744b8dd_170c_433e_b137_c5bdbc2ab252.slice/crio-7421a7902f804733cb8601572852fa6ded2e85972876f01abca9f2cb0e582162 WatchSource:0}: Error finding container 7421a7902f804733cb8601572852fa6ded2e85972876f01abca9f2cb0e582162: Status 404 returned error can't find the container with id 7421a7902f804733cb8601572852fa6ded2e85972876f01abca9f2cb0e582162 Jun 11 18:41:43.659260 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:43.659219 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-scn5g" event={"ID":"2744b8dd-170c-433e-b137-c5bdbc2ab252","Type":"ContainerStarted","Data":"7421a7902f804733cb8601572852fa6ded2e85972876f01abca9f2cb0e582162"} Jun 11 18:41:45.669714 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:45.669678 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8d878499b-92tdm" event={"ID":"970e8ea0-a08b-46ea-badb-f9d914f344a1","Type":"ContainerStarted","Data":"f43a69e06a77dd05728373a7d8651abacb645b63b1b609920add5cf5f16eaff0"} Jun 11 18:41:45.685749 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:45.685703 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8d878499b-92tdm" podStartSLOduration=1.453766056 podStartE2EDuration="4.685688926s" totalImagesPullingTime="3.23192287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:41:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:41:45.684273657 +0000 UTC m=+347.252347174" watchObservedRunningTime="2026-06-11 18:41:45.685688926 +0000 UTC m=+347.253762405" Jun 11 18:41:46.675451 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:46.675360 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-scn5g" event={"ID":"2744b8dd-170c-433e-b137-c5bdbc2ab252","Type":"ContainerStarted","Data":"723dfa244aebceb898fb8280d66fec637fb8a29ec4f87f359c753ad985d50265"} Jun 11 18:41:46.675816 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:46.675648 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-5dd488fc48-scn5g" Jun 11 18:41:46.694153 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:46.694096 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-5dd488fc48-scn5g" podStartSLOduration=1.711319385 podStartE2EDuration="4.69408158s" totalImagesPullingTime="2.982762195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:41:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:41:46.692222692 +0000 UTC m=+348.260296173" watchObservedRunningTime="2026-06-11 18:41:46.69408158 +0000 UTC m=+348.262155060" Jun 11 18:41:51.838607 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:51.838559 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-zgv4d"] Jun 11 18:41:51.842278 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:51.842257 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-zgv4d" Jun 11 18:41:51.844778 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:51.844757 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 11 18:41:51.844902 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:51.844802 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-74prh\"" Jun 11 18:41:51.854832 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:51.854806 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-zgv4d"] Jun 11 18:41:51.937790 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:51.937750 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2kdt\" (UniqueName: \"kubernetes.io/projected/31721fa7-6100-41c0-a64c-62498ab0e81a-kube-api-access-v2kdt\") pod \"kserve-controller-manager-69fc866d76-zgv4d\" (UID: \"31721fa7-6100-41c0-a64c-62498ab0e81a\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zgv4d" Jun 11 18:41:51.937790 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:51.937790 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31721fa7-6100-41c0-a64c-62498ab0e81a-cert\") pod \"kserve-controller-manager-69fc866d76-zgv4d\" (UID: \"31721fa7-6100-41c0-a64c-62498ab0e81a\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zgv4d" Jun 11 18:41:52.039142 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:52.039104 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2kdt\" (UniqueName: \"kubernetes.io/projected/31721fa7-6100-41c0-a64c-62498ab0e81a-kube-api-access-v2kdt\") pod \"kserve-controller-manager-69fc866d76-zgv4d\" (UID: \"31721fa7-6100-41c0-a64c-62498ab0e81a\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zgv4d" Jun 11 18:41:52.039142 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:52.039143 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31721fa7-6100-41c0-a64c-62498ab0e81a-cert\") pod \"kserve-controller-manager-69fc866d76-zgv4d\" (UID: \"31721fa7-6100-41c0-a64c-62498ab0e81a\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zgv4d" Jun 11 18:41:52.039407 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:41:52.039271 2561 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jun 11 18:41:52.039407 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:41:52.039323 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31721fa7-6100-41c0-a64c-62498ab0e81a-cert podName:31721fa7-6100-41c0-a64c-62498ab0e81a nodeName:}" failed. No retries permitted until 2026-06-11 18:41:52.539309213 +0000 UTC m=+354.107382672 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/31721fa7-6100-41c0-a64c-62498ab0e81a-cert") pod "kserve-controller-manager-69fc866d76-zgv4d" (UID: "31721fa7-6100-41c0-a64c-62498ab0e81a") : secret "kserve-webhook-server-cert" not found Jun 11 18:41:52.054859 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:52.054825 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2kdt\" (UniqueName: \"kubernetes.io/projected/31721fa7-6100-41c0-a64c-62498ab0e81a-kube-api-access-v2kdt\") pod \"kserve-controller-manager-69fc866d76-zgv4d\" (UID: \"31721fa7-6100-41c0-a64c-62498ab0e81a\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zgv4d" Jun 11 18:41:52.543610 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:52.543565 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31721fa7-6100-41c0-a64c-62498ab0e81a-cert\") pod \"kserve-controller-manager-69fc866d76-zgv4d\" (UID: \"31721fa7-6100-41c0-a64c-62498ab0e81a\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zgv4d" Jun 11 18:41:52.545987 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:52.545961 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31721fa7-6100-41c0-a64c-62498ab0e81a-cert\") pod \"kserve-controller-manager-69fc866d76-zgv4d\" (UID: \"31721fa7-6100-41c0-a64c-62498ab0e81a\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zgv4d" Jun 11 18:41:52.753827 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:52.753787 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-zgv4d" Jun 11 18:41:52.890500 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:52.890474 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-zgv4d"] Jun 11 18:41:52.892687 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:41:52.892654 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31721fa7_6100_41c0_a64c_62498ab0e81a.slice/crio-157ea91d8a68d8b5c1fb11bb85b0629054108c78de8a6b8d1f83fe87eb19cf18 WatchSource:0}: Error finding container 157ea91d8a68d8b5c1fb11bb85b0629054108c78de8a6b8d1f83fe87eb19cf18: Status 404 returned error can't find the container with id 157ea91d8a68d8b5c1fb11bb85b0629054108c78de8a6b8d1f83fe87eb19cf18 Jun 11 18:41:53.706548 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:53.706514 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-zgv4d" event={"ID":"31721fa7-6100-41c0-a64c-62498ab0e81a","Type":"ContainerStarted","Data":"157ea91d8a68d8b5c1fb11bb85b0629054108c78de8a6b8d1f83fe87eb19cf18"} Jun 11 18:41:56.720454 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:56.720417 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-zgv4d" event={"ID":"31721fa7-6100-41c0-a64c-62498ab0e81a","Type":"ContainerStarted","Data":"07e9bc49ff5190ba536408612b940a43d8d297a02b2587d9b8cac29f9bd901e1"} Jun 11 18:41:56.720852 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:56.720605 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-69fc866d76-zgv4d" Jun 11 18:41:56.736807 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:56.736756 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-69fc866d76-zgv4d" podStartSLOduration=2.947175355 podStartE2EDuration="5.736739597s" totalImagesPullingTime="2.789564242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:41:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:41:56.735738794 +0000 UTC m=+358.303812275" watchObservedRunningTime="2026-06-11 18:41:56.736739597 +0000 UTC m=+358.304813076" Jun 11 18:41:57.683624 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:41:57.683592 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-5dd488fc48-scn5g" Jun 11 18:42:10.830873 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.830839 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4"] Jun 11 18:42:10.835871 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.835850 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.841471 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.841442 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8wdsb\"" Jun 11 18:42:10.841614 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.841442 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 11 18:42:10.841614 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.841484 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 11 18:42:10.841614 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.841445 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 11 18:42:10.841614 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.841471 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 11 18:42:10.848354 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.848331 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4"] Jun 11 18:42:10.893417 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.893387 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/241dca3f-d650-4f38-94c0-5b0b94a17619-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.893417 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.893415 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/241dca3f-d650-4f38-94c0-5b0b94a17619-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.893667 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.893445 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/241dca3f-d650-4f38-94c0-5b0b94a17619-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.893667 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.893499 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/241dca3f-d650-4f38-94c0-5b0b94a17619-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.893667 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.893544 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/241dca3f-d650-4f38-94c0-5b0b94a17619-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.893667 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.893576 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/241dca3f-d650-4f38-94c0-5b0b94a17619-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.893667 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.893624 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksktp\" (UniqueName: \"kubernetes.io/projected/241dca3f-d650-4f38-94c0-5b0b94a17619-kube-api-access-ksktp\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.994511 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.994463 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/241dca3f-d650-4f38-94c0-5b0b94a17619-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.994511 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.994517 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/241dca3f-d650-4f38-94c0-5b0b94a17619-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.994830 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.994547 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/241dca3f-d650-4f38-94c0-5b0b94a17619-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.994830 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.994572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/241dca3f-d650-4f38-94c0-5b0b94a17619-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.994830 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.994611 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksktp\" (UniqueName: \"kubernetes.io/projected/241dca3f-d650-4f38-94c0-5b0b94a17619-kube-api-access-ksktp\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.994830 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.994759 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/241dca3f-d650-4f38-94c0-5b0b94a17619-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.994830 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.994801 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/241dca3f-d650-4f38-94c0-5b0b94a17619-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.995345 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.995319 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/241dca3f-d650-4f38-94c0-5b0b94a17619-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.996902 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.996873 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/241dca3f-d650-4f38-94c0-5b0b94a17619-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.997021 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.996978 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/241dca3f-d650-4f38-94c0-5b0b94a17619-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.997089 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.997044 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/241dca3f-d650-4f38-94c0-5b0b94a17619-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:10.997089 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:10.997074 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/241dca3f-d650-4f38-94c0-5b0b94a17619-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:11.002966 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:11.002943 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/241dca3f-d650-4f38-94c0-5b0b94a17619-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:11.003866 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:11.003844 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksktp\" (UniqueName: \"kubernetes.io/projected/241dca3f-d650-4f38-94c0-5b0b94a17619-kube-api-access-ksktp\") pod \"istiod-openshift-gateway-75c67f8887-wlhf4\" (UID: \"241dca3f-d650-4f38-94c0-5b0b94a17619\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:11.144853 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:11.144757 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:11.283354 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:11.283319 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4"] Jun 11 18:42:11.287447 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:42:11.287420 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241dca3f_d650_4f38_94c0_5b0b94a17619.slice/crio-e54c01717f20a3d10d850802ee6d18be0affa3e65c4eb6ab0c9b7898b5d91369 WatchSource:0}: Error finding container e54c01717f20a3d10d850802ee6d18be0affa3e65c4eb6ab0c9b7898b5d91369: Status 404 returned error can't find the container with id e54c01717f20a3d10d850802ee6d18be0affa3e65c4eb6ab0c9b7898b5d91369 Jun 11 18:42:11.792538 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:11.792489 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" event={"ID":"241dca3f-d650-4f38-94c0-5b0b94a17619","Type":"ContainerStarted","Data":"e54c01717f20a3d10d850802ee6d18be0affa3e65c4eb6ab0c9b7898b5d91369"} Jun 11 18:42:13.896879 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:13.896831 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 18:42:13.897194 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:13.896909 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 18:42:14.810497 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:14.810451 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" event={"ID":"241dca3f-d650-4f38-94c0-5b0b94a17619","Type":"ContainerStarted","Data":"2cdd67942d1c53e8c8db942b38da0b7ac3b0d58e910a819bd860c42fc752f737"} Jun 11 18:42:14.810703 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:14.810637 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:14.812227 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:14.812201 2561 patch_prober.go:28] interesting pod/istiod-openshift-gateway-75c67f8887-wlhf4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 11 18:42:14.812339 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:14.812245 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" podUID="241dca3f-d650-4f38-94c0-5b0b94a17619" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 18:42:14.830595 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:14.830552 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" podStartSLOduration=2.224006987 podStartE2EDuration="4.830537298s" totalImagesPullingTime="2.606530311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:42:14.829214631 +0000 UTC m=+376.397288130" watchObservedRunningTime="2026-06-11 18:42:14.830537298 +0000 UTC m=+376.398610778" Jun 11 18:42:15.815516 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:15.815485 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-wlhf4" Jun 11 18:42:27.730932 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:42:27.730903 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-69fc866d76-zgv4d" Jun 11 18:43:43.475381 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:43.475348 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9"] Jun 11 18:43:43.478832 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:43.478813 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9" Jun 11 18:43:43.481340 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:43.481311 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 11 18:43:43.481745 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:43.481725 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 11 18:43:43.481919 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:43.481905 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4xmkb\"" Jun 11 18:43:43.491329 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:43.491303 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9"] Jun 11 18:43:43.574965 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:43.574934 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9"] Jun 11 18:43:43.575236 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:43:43.575215 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-j49v6], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9" podUID="6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9" Jun 11 18:43:43.581231 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:43.581205 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6ndc9\" (UID: \"6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9" Jun 11 18:43:43.581338 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:43.581241 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j49v6\" (UniqueName: \"kubernetes.io/projected/6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9-kube-api-access-j49v6\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6ndc9\" (UID: \"6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9" Jun 11 18:43:43.682390 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:43.682273 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6ndc9\" (UID: \"6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9" Jun 11 18:43:43.682390 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:43.682325 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j49v6\" (UniqueName: \"kubernetes.io/projected/6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9-kube-api-access-j49v6\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6ndc9\" (UID: \"6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9" Jun 11 18:43:43.682771 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:43.682747 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6ndc9\" (UID: \"6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9" Jun 11 18:43:43.691819 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:43.691795 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j49v6\" (UniqueName: \"kubernetes.io/projected/6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9-kube-api-access-j49v6\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6ndc9\" (UID: \"6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9" Jun 11 18:43:44.142109 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:44.142072 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9"] Jun 11 18:43:44.149311 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:44.149283 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9"] Jun 11 18:43:44.225007 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:44.224976 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9" Jun 11 18:43:44.227498 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:43:44.227467 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-6ndc9\" is forbidden: User \"system:node:ip-10-0-138-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-156.ec2.internal' and this object" podUID="6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9" Jun 11 18:43:44.232660 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:44.232638 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9" Jun 11 18:43:44.234373 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:43:44.234344 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-6ndc9\" is forbidden: User \"system:node:ip-10-0-138-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-156.ec2.internal' and this object" podUID="6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9" Jun 11 18:43:44.389063 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:44.389025 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j49v6\" (UniqueName: \"kubernetes.io/projected/6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9-kube-api-access-j49v6\") pod \"6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9\" (UID: \"6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9\") " Jun 11 18:43:44.389266 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:44.389161 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9-extensions-socket-volume\") pod \"6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9\" (UID: \"6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9\") " Jun 11 18:43:44.389444 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:44.389414 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9" (UID: "6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 18:43:44.391237 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:44.391210 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9-kube-api-access-j49v6" (OuterVolumeSpecName: "kube-api-access-j49v6") pod "6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9" (UID: "6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9"). InnerVolumeSpecName "kube-api-access-j49v6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 18:43:44.490741 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:44.490700 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9-extensions-socket-volume\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:43:44.490741 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:44.490734 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j49v6\" (UniqueName: \"kubernetes.io/projected/6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9-kube-api-access-j49v6\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:43:44.919150 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:44.919114 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9" path="/var/lib/kubelet/pods/6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9/volumes" Jun 11 18:43:45.229784 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:43:45.229706 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9" Jun 11 18:43:45.235509 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:43:45.235477 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-6ndc9\" is forbidden: User \"system:node:ip-10-0-138-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-156.ec2.internal' and this object" podUID="6ded3c8b-67ec-44ad-95b4-2bffc8bb23f9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6ndc9" Jun 11 18:44:40.497548 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.497515 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qxxcf"] Jun 11 18:44:40.500948 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.500924 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" Jun 11 18:44:40.503522 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.503493 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 11 18:44:40.503642 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.503537 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 11 18:44:40.504003 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.503990 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 11 18:44:40.504051 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.504024 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pz7qv\"" Jun 11 18:44:40.508059 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.508036 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qxxcf"] Jun 11 18:44:40.572592 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.572557 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e24d4c33-379d-4a97-969c-41096dbe1147-config-file\") pod \"limitador-limitador-64c94fd64f-qxxcf\" (UID: \"e24d4c33-379d-4a97-969c-41096dbe1147\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" Jun 11 18:44:40.572757 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.572627 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v44t4\" (UniqueName: \"kubernetes.io/projected/e24d4c33-379d-4a97-969c-41096dbe1147-kube-api-access-v44t4\") pod \"limitador-limitador-64c94fd64f-qxxcf\" (UID: \"e24d4c33-379d-4a97-969c-41096dbe1147\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" Jun 11 18:44:40.602372 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.602336 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qxxcf"] Jun 11 18:44:40.673436 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.673404 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v44t4\" (UniqueName: \"kubernetes.io/projected/e24d4c33-379d-4a97-969c-41096dbe1147-kube-api-access-v44t4\") pod \"limitador-limitador-64c94fd64f-qxxcf\" (UID: \"e24d4c33-379d-4a97-969c-41096dbe1147\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" Jun 11 18:44:40.673633 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.673486 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e24d4c33-379d-4a97-969c-41096dbe1147-config-file\") pod \"limitador-limitador-64c94fd64f-qxxcf\" (UID: \"e24d4c33-379d-4a97-969c-41096dbe1147\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" Jun 11 18:44:40.674070 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.674052 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e24d4c33-379d-4a97-969c-41096dbe1147-config-file\") pod \"limitador-limitador-64c94fd64f-qxxcf\" (UID: \"e24d4c33-379d-4a97-969c-41096dbe1147\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" Jun 11 18:44:40.682794 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.682767 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v44t4\" (UniqueName: \"kubernetes.io/projected/e24d4c33-379d-4a97-969c-41096dbe1147-kube-api-access-v44t4\") pod \"limitador-limitador-64c94fd64f-qxxcf\" (UID: \"e24d4c33-379d-4a97-969c-41096dbe1147\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" Jun 11 18:44:40.811840 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.811751 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" Jun 11 18:44:40.946936 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:40.946911 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qxxcf"] Jun 11 18:44:40.949291 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:44:40.949262 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode24d4c33_379d_4a97_969c_41096dbe1147.slice/crio-9bd38de39232c4a24ba954865b4fbbdb13fca04ca86b337959578f6816e1c8ad WatchSource:0}: Error finding container 9bd38de39232c4a24ba954865b4fbbdb13fca04ca86b337959578f6816e1c8ad: Status 404 returned error can't find the container with id 9bd38de39232c4a24ba954865b4fbbdb13fca04ca86b337959578f6816e1c8ad Jun 11 18:44:41.489978 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:41.489939 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" event={"ID":"e24d4c33-379d-4a97-969c-41096dbe1147","Type":"ContainerStarted","Data":"9bd38de39232c4a24ba954865b4fbbdb13fca04ca86b337959578f6816e1c8ad"} Jun 11 18:44:44.506471 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:44.506437 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" event={"ID":"e24d4c33-379d-4a97-969c-41096dbe1147","Type":"ContainerStarted","Data":"116bfb3c961b3286e39e044d2d336855950f8197859d54fed273ae7fc8c50980"} Jun 11 18:44:44.506930 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:44.506561 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" Jun 11 18:44:44.524092 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:44.524040 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" podStartSLOduration=1.78908105 podStartE2EDuration="4.524024846s" totalImagesPullingTime="2.734943796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:44:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:44:44.522457212 +0000 UTC m=+526.090530715" watchObservedRunningTime="2026-06-11 18:44:44.524024846 +0000 UTC m=+526.092098326" Jun 11 18:44:55.512773 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:55.512678 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" Jun 11 18:44:55.814238 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:55.814118 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qxxcf"] Jun 11 18:44:55.814394 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:55.814342 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" podUID="e24d4c33-379d-4a97-969c-41096dbe1147" containerName="limitador" containerID="cri-o://116bfb3c961b3286e39e044d2d336855950f8197859d54fed273ae7fc8c50980" gracePeriod=30 Jun 11 18:44:56.359663 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.359639 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" Jun 11 18:44:56.410232 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.410147 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v44t4\" (UniqueName: \"kubernetes.io/projected/e24d4c33-379d-4a97-969c-41096dbe1147-kube-api-access-v44t4\") pod \"e24d4c33-379d-4a97-969c-41096dbe1147\" (UID: \"e24d4c33-379d-4a97-969c-41096dbe1147\") " Jun 11 18:44:56.410232 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.410209 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e24d4c33-379d-4a97-969c-41096dbe1147-config-file\") pod \"e24d4c33-379d-4a97-969c-41096dbe1147\" (UID: \"e24d4c33-379d-4a97-969c-41096dbe1147\") " Jun 11 18:44:56.410642 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.410608 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e24d4c33-379d-4a97-969c-41096dbe1147-config-file" (OuterVolumeSpecName: "config-file") pod "e24d4c33-379d-4a97-969c-41096dbe1147" (UID: "e24d4c33-379d-4a97-969c-41096dbe1147"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 18:44:56.412330 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.412298 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e24d4c33-379d-4a97-969c-41096dbe1147-kube-api-access-v44t4" (OuterVolumeSpecName: "kube-api-access-v44t4") pod "e24d4c33-379d-4a97-969c-41096dbe1147" (UID: "e24d4c33-379d-4a97-969c-41096dbe1147"). InnerVolumeSpecName "kube-api-access-v44t4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 18:44:56.511068 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.511032 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v44t4\" (UniqueName: \"kubernetes.io/projected/e24d4c33-379d-4a97-969c-41096dbe1147-kube-api-access-v44t4\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:44:56.511068 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.511062 2561 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e24d4c33-379d-4a97-969c-41096dbe1147-config-file\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 18:44:56.563058 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.563021 2561 generic.go:356] "Generic (PLEG): container finished" podID="e24d4c33-379d-4a97-969c-41096dbe1147" containerID="116bfb3c961b3286e39e044d2d336855950f8197859d54fed273ae7fc8c50980" exitCode=0 Jun 11 18:44:56.563503 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.563065 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" Jun 11 18:44:56.563503 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.563095 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" event={"ID":"e24d4c33-379d-4a97-969c-41096dbe1147","Type":"ContainerDied","Data":"116bfb3c961b3286e39e044d2d336855950f8197859d54fed273ae7fc8c50980"} Jun 11 18:44:56.563503 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.563142 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxxcf" event={"ID":"e24d4c33-379d-4a97-969c-41096dbe1147","Type":"ContainerDied","Data":"9bd38de39232c4a24ba954865b4fbbdb13fca04ca86b337959578f6816e1c8ad"} Jun 11 18:44:56.563503 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.563189 2561 scope.go:117] "RemoveContainer" containerID="116bfb3c961b3286e39e044d2d336855950f8197859d54fed273ae7fc8c50980" Jun 11 18:44:56.575308 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.575283 2561 scope.go:117] "RemoveContainer" containerID="116bfb3c961b3286e39e044d2d336855950f8197859d54fed273ae7fc8c50980" Jun 11 18:44:56.575568 ip-10-0-138-156 kubenswrapper[2561]: E0611 18:44:56.575549 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"116bfb3c961b3286e39e044d2d336855950f8197859d54fed273ae7fc8c50980\": container with ID starting with 116bfb3c961b3286e39e044d2d336855950f8197859d54fed273ae7fc8c50980 not found: ID does not exist" containerID="116bfb3c961b3286e39e044d2d336855950f8197859d54fed273ae7fc8c50980" Jun 11 18:44:56.575611 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.575578 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"116bfb3c961b3286e39e044d2d336855950f8197859d54fed273ae7fc8c50980"} err="failed to get container status \"116bfb3c961b3286e39e044d2d336855950f8197859d54fed273ae7fc8c50980\": rpc error: code = NotFound desc = could not find container \"116bfb3c961b3286e39e044d2d336855950f8197859d54fed273ae7fc8c50980\": container with ID starting with 116bfb3c961b3286e39e044d2d336855950f8197859d54fed273ae7fc8c50980 not found: ID does not exist" Jun 11 18:44:56.587490 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.587466 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qxxcf"] Jun 11 18:44:56.591536 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.591513 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qxxcf"] Jun 11 18:44:56.918597 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:56.918563 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e24d4c33-379d-4a97-969c-41096dbe1147" path="/var/lib/kubelet/pods/e24d4c33-379d-4a97-969c-41096dbe1147/volumes" Jun 11 18:44:59.284624 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.284546 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-z9pbp"] Jun 11 18:44:59.284954 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.284899 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e24d4c33-379d-4a97-969c-41096dbe1147" containerName="limitador" Jun 11 18:44:59.284954 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.284911 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e24d4c33-379d-4a97-969c-41096dbe1147" containerName="limitador" Jun 11 18:44:59.285029 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.284985 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e24d4c33-379d-4a97-969c-41096dbe1147" containerName="limitador" Jun 11 18:44:59.289214 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.289193 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-z9pbp" Jun 11 18:44:59.291311 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.291283 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 11 18:44:59.291423 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.291364 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8n27r\"" Jun 11 18:44:59.294959 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.294929 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-z9pbp"] Jun 11 18:44:59.333987 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.333948 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgqht\" (UniqueName: \"kubernetes.io/projected/dd4f0814-5848-4adf-b334-e9cf64d511cb-kube-api-access-vgqht\") pod \"postgres-69fb4594cf-z9pbp\" (UID: \"dd4f0814-5848-4adf-b334-e9cf64d511cb\") " pod="opendatahub/postgres-69fb4594cf-z9pbp" Jun 11 18:44:59.334216 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.334017 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dd4f0814-5848-4adf-b334-e9cf64d511cb-data\") pod \"postgres-69fb4594cf-z9pbp\" (UID: \"dd4f0814-5848-4adf-b334-e9cf64d511cb\") " pod="opendatahub/postgres-69fb4594cf-z9pbp" Jun 11 18:44:59.435144 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.435100 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgqht\" (UniqueName: \"kubernetes.io/projected/dd4f0814-5848-4adf-b334-e9cf64d511cb-kube-api-access-vgqht\") pod \"postgres-69fb4594cf-z9pbp\" (UID: \"dd4f0814-5848-4adf-b334-e9cf64d511cb\") " pod="opendatahub/postgres-69fb4594cf-z9pbp" Jun 11 18:44:59.435352 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.435225 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dd4f0814-5848-4adf-b334-e9cf64d511cb-data\") pod \"postgres-69fb4594cf-z9pbp\" (UID: \"dd4f0814-5848-4adf-b334-e9cf64d511cb\") " pod="opendatahub/postgres-69fb4594cf-z9pbp" Jun 11 18:44:59.435593 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.435573 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dd4f0814-5848-4adf-b334-e9cf64d511cb-data\") pod \"postgres-69fb4594cf-z9pbp\" (UID: \"dd4f0814-5848-4adf-b334-e9cf64d511cb\") " pod="opendatahub/postgres-69fb4594cf-z9pbp" Jun 11 18:44:59.445557 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.445527 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgqht\" (UniqueName: \"kubernetes.io/projected/dd4f0814-5848-4adf-b334-e9cf64d511cb-kube-api-access-vgqht\") pod \"postgres-69fb4594cf-z9pbp\" (UID: \"dd4f0814-5848-4adf-b334-e9cf64d511cb\") " pod="opendatahub/postgres-69fb4594cf-z9pbp" Jun 11 18:44:59.601768 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.601733 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-z9pbp" Jun 11 18:44:59.728531 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:44:59.728375 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-z9pbp"] Jun 11 18:44:59.731141 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:44:59.731116 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd4f0814_5848_4adf_b334_e9cf64d511cb.slice/crio-c53163513bd6d4cee3e7b7e11353e6c194479ff9ca6ba3020595dd5f3479f84b WatchSource:0}: Error finding container c53163513bd6d4cee3e7b7e11353e6c194479ff9ca6ba3020595dd5f3479f84b: Status 404 returned error can't find the container with id c53163513bd6d4cee3e7b7e11353e6c194479ff9ca6ba3020595dd5f3479f84b Jun 11 18:45:00.584600 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:00.584555 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-z9pbp" event={"ID":"dd4f0814-5848-4adf-b334-e9cf64d511cb","Type":"ContainerStarted","Data":"c53163513bd6d4cee3e7b7e11353e6c194479ff9ca6ba3020595dd5f3479f84b"} Jun 11 18:45:02.595069 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:02.595033 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-z9pbp" event={"ID":"dd4f0814-5848-4adf-b334-e9cf64d511cb","Type":"ContainerStarted","Data":"6841698bd174eeacac8c82c9d9d2697c9f5108396a2c9b70b49528e38b5d3a3a"} Jun 11 18:45:02.595471 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:02.595106 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-z9pbp" Jun 11 18:45:02.611085 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:02.611034 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-z9pbp" podStartSLOduration=1.3791261430000001 podStartE2EDuration="3.611017425s" totalImagesPullingTime="2.231891282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:44:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:45:02.610020884 +0000 UTC m=+544.178094364" watchObservedRunningTime="2026-06-11 18:45:02.611017425 +0000 UTC m=+544.179090905" Jun 11 18:45:08.632089 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:08.632056 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-z9pbp" Jun 11 18:45:25.188558 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:25.188522 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-fh6r7"] Jun 11 18:45:25.191934 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:25.191915 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-fh6r7" Jun 11 18:45:25.194654 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:25.194630 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 11 18:45:25.195471 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:25.195451 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"keycloak-operator-dockercfg-8zwnx\"" Jun 11 18:45:25.195567 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:25.195491 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 11 18:45:25.202394 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:25.202364 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-fh6r7"] Jun 11 18:45:25.269853 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:25.269812 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmkk7\" (UniqueName: \"kubernetes.io/projected/224352e2-0214-4f04-a767-b443c5c58546-kube-api-access-bmkk7\") pod \"keycloak-operator-d67dbbb48-fh6r7\" (UID: \"224352e2-0214-4f04-a767-b443c5c58546\") " pod="keycloak-system/keycloak-operator-d67dbbb48-fh6r7" Jun 11 18:45:25.371274 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:25.371232 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmkk7\" (UniqueName: \"kubernetes.io/projected/224352e2-0214-4f04-a767-b443c5c58546-kube-api-access-bmkk7\") pod \"keycloak-operator-d67dbbb48-fh6r7\" (UID: \"224352e2-0214-4f04-a767-b443c5c58546\") " pod="keycloak-system/keycloak-operator-d67dbbb48-fh6r7" Jun 11 18:45:25.385024 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:25.384986 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmkk7\" (UniqueName: \"kubernetes.io/projected/224352e2-0214-4f04-a767-b443c5c58546-kube-api-access-bmkk7\") pod \"keycloak-operator-d67dbbb48-fh6r7\" (UID: \"224352e2-0214-4f04-a767-b443c5c58546\") " pod="keycloak-system/keycloak-operator-d67dbbb48-fh6r7" Jun 11 18:45:25.503359 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:25.503268 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-fh6r7" Jun 11 18:45:25.644096 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:25.643916 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-fh6r7"] Jun 11 18:45:25.647007 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:45:25.646981 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod224352e2_0214_4f04_a767_b443c5c58546.slice/crio-027de070cc27b8ac167fdb7bc5af2564580073a78d1ffabbc1c43776319c6602 WatchSource:0}: Error finding container 027de070cc27b8ac167fdb7bc5af2564580073a78d1ffabbc1c43776319c6602: Status 404 returned error can't find the container with id 027de070cc27b8ac167fdb7bc5af2564580073a78d1ffabbc1c43776319c6602 Jun 11 18:45:25.697857 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:25.697822 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-fh6r7" event={"ID":"224352e2-0214-4f04-a767-b443c5c58546","Type":"ContainerStarted","Data":"027de070cc27b8ac167fdb7bc5af2564580073a78d1ffabbc1c43776319c6602"} Jun 11 18:45:32.738688 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:32.738643 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-fh6r7" event={"ID":"224352e2-0214-4f04-a767-b443c5c58546","Type":"ContainerStarted","Data":"18dcb0c360dc2c3a6a5cebdff4ac8158fe001532949772df5d8a334002eb135c"} Jun 11 18:45:32.754650 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:32.754593 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/keycloak-operator-d67dbbb48-fh6r7" podStartSLOduration=1.711654346 podStartE2EDuration="7.754577312s" totalImagesPullingTime="6.042922966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:45:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:45:32.753015408 +0000 UTC m=+574.321088920" watchObservedRunningTime="2026-06-11 18:45:32.754577312 +0000 UTC m=+574.322650792" Jun 11 18:45:58.890195 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:58.890145 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 18:45:58.890719 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:58.890251 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 18:45:58.896713 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:58.896691 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 18:45:58.896814 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:45:58.896692 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 18:46:51.089667 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:46:51.089634 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-jghh4"] Jun 11 18:46:51.092952 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:46:51.092932 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-jghh4" Jun 11 18:46:51.095376 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:46:51.095352 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 11 18:46:51.103527 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:46:51.103497 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-jghh4"] Jun 11 18:46:51.215318 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:46:51.215280 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrqbs\" (UniqueName: \"kubernetes.io/projected/e6180710-f1b3-4c0a-8d79-4ca0c7dd7245-kube-api-access-xrqbs\") pod \"payload-processing-699b6cd6c-jghh4\" (UID: \"e6180710-f1b3-4c0a-8d79-4ca0c7dd7245\") " pod="openshift-ingress/payload-processing-699b6cd6c-jghh4" Jun 11 18:46:51.316331 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:46:51.316294 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrqbs\" (UniqueName: \"kubernetes.io/projected/e6180710-f1b3-4c0a-8d79-4ca0c7dd7245-kube-api-access-xrqbs\") pod \"payload-processing-699b6cd6c-jghh4\" (UID: \"e6180710-f1b3-4c0a-8d79-4ca0c7dd7245\") " pod="openshift-ingress/payload-processing-699b6cd6c-jghh4" Jun 11 18:46:51.330597 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:46:51.330565 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrqbs\" (UniqueName: \"kubernetes.io/projected/e6180710-f1b3-4c0a-8d79-4ca0c7dd7245-kube-api-access-xrqbs\") pod \"payload-processing-699b6cd6c-jghh4\" (UID: \"e6180710-f1b3-4c0a-8d79-4ca0c7dd7245\") " pod="openshift-ingress/payload-processing-699b6cd6c-jghh4" Jun 11 18:46:51.404596 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:46:51.404502 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-jghh4" Jun 11 18:46:51.532287 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:46:51.532264 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-jghh4"] Jun 11 18:46:51.535059 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:46:51.535025 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6180710_f1b3_4c0a_8d79_4ca0c7dd7245.slice/crio-86a30987ec14f5b251312a2654113b71d67f6ff626ec4d99c402666f05c341a4 WatchSource:0}: Error finding container 86a30987ec14f5b251312a2654113b71d67f6ff626ec4d99c402666f05c341a4: Status 404 returned error can't find the container with id 86a30987ec14f5b251312a2654113b71d67f6ff626ec4d99c402666f05c341a4 Jun 11 18:46:51.536883 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:46:51.536860 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 18:46:52.098924 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:46:52.098888 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-jghh4" event={"ID":"e6180710-f1b3-4c0a-8d79-4ca0c7dd7245","Type":"ContainerStarted","Data":"86a30987ec14f5b251312a2654113b71d67f6ff626ec4d99c402666f05c341a4"} Jun 11 18:46:54.111649 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:46:54.111616 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-jghh4" event={"ID":"e6180710-f1b3-4c0a-8d79-4ca0c7dd7245","Type":"ContainerStarted","Data":"1727522057a4918ddf0dd88db9b7837825b28f416441c72b5dbc012cc61e7fb0"} Jun 11 18:46:54.112045 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:46:54.111745 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-699b6cd6c-jghh4" Jun 11 18:46:54.130692 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:46:54.130626 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-699b6cd6c-jghh4" podStartSLOduration=0.877777097 podStartE2EDuration="3.130608859s" totalImagesPullingTime="2.252831762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:46:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:46:54.129845295 +0000 UTC m=+655.697918785" watchObservedRunningTime="2026-06-11 18:46:54.130608859 +0000 UTC m=+655.698682340" Jun 11 18:47:05.119778 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:47:05.119743 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-699b6cd6c-jghh4" Jun 11 18:50:07.330679 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.330634 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7"] Jun 11 18:50:07.334319 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.334299 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.336592 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.336564 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 11 18:50:07.336709 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.336592 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 11 18:50:07.336775 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.336718 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 11 18:50:07.336929 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.336909 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bxzds\"" Jun 11 18:50:07.347268 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.347241 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7"] Jun 11 18:50:07.463455 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.463419 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jldct\" (UniqueName: \"kubernetes.io/projected/0caea8f3-61a6-480b-8885-4ec8561b6f5d-kube-api-access-jldct\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.463631 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.463516 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0caea8f3-61a6-480b-8885-4ec8561b6f5d-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.463631 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.463535 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0caea8f3-61a6-480b-8885-4ec8561b6f5d-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.463631 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.463588 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0caea8f3-61a6-480b-8885-4ec8561b6f5d-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.463631 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.463603 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0caea8f3-61a6-480b-8885-4ec8561b6f5d-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.463631 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.463622 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0caea8f3-61a6-480b-8885-4ec8561b6f5d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.564451 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.564413 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0caea8f3-61a6-480b-8885-4ec8561b6f5d-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.564451 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.564448 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0caea8f3-61a6-480b-8885-4ec8561b6f5d-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.564651 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.564467 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0caea8f3-61a6-480b-8885-4ec8561b6f5d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.564651 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.564499 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jldct\" (UniqueName: \"kubernetes.io/projected/0caea8f3-61a6-480b-8885-4ec8561b6f5d-kube-api-access-jldct\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.564651 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.564594 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0caea8f3-61a6-480b-8885-4ec8561b6f5d-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.564651 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.564613 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0caea8f3-61a6-480b-8885-4ec8561b6f5d-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.564900 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.564860 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0caea8f3-61a6-480b-8885-4ec8561b6f5d-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.565021 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.564904 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0caea8f3-61a6-480b-8885-4ec8561b6f5d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.565021 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.564927 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0caea8f3-61a6-480b-8885-4ec8561b6f5d-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.566856 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.566838 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0caea8f3-61a6-480b-8885-4ec8561b6f5d-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.567101 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.567084 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0caea8f3-61a6-480b-8885-4ec8561b6f5d-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.571902 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.571883 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jldct\" (UniqueName: \"kubernetes.io/projected/0caea8f3-61a6-480b-8885-4ec8561b6f5d-kube-api-access-jldct\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-58sj7\" (UID: \"0caea8f3-61a6-480b-8885-4ec8561b6f5d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.644768 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.644673 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:50:07.790778 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:07.790754 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7"] Jun 11 18:50:07.792995 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:50:07.792964 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0caea8f3_61a6_480b_8885_4ec8561b6f5d.slice/crio-84364cf180b08014d9a4c708a09b175f14eddf87137508689e88dcf834a370a5 WatchSource:0}: Error finding container 84364cf180b08014d9a4c708a09b175f14eddf87137508689e88dcf834a370a5: Status 404 returned error can't find the container with id 84364cf180b08014d9a4c708a09b175f14eddf87137508689e88dcf834a370a5 Jun 11 18:50:08.025746 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:08.025654 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" event={"ID":"0caea8f3-61a6-480b-8885-4ec8561b6f5d","Type":"ContainerStarted","Data":"84364cf180b08014d9a4c708a09b175f14eddf87137508689e88dcf834a370a5"} Jun 11 18:50:13.051566 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:13.051523 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" event={"ID":"0caea8f3-61a6-480b-8885-4ec8561b6f5d","Type":"ContainerStarted","Data":"82b969541b3639c63bc8b4a33751d7c567713be96825e664bcd82df5f5e580f7"} Jun 11 18:50:39.657095 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.657063 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j"] Jun 11 18:50:39.677035 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.677008 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j"] Jun 11 18:50:39.677210 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.677109 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.679529 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.679505 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 11 18:50:39.752494 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.752464 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.752614 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.752506 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzrqb\" (UniqueName: \"kubernetes.io/projected/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-kube-api-access-bzrqb\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.752658 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.752605 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.752698 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.752663 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.752739 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.752709 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.752779 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.752742 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.854127 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.854085 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.854127 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.854133 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzrqb\" (UniqueName: \"kubernetes.io/projected/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-kube-api-access-bzrqb\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.854397 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.854208 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.854397 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.854238 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.854397 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.854270 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.854397 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.854291 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.854602 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.854510 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.854681 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.854651 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.854722 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.854688 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.856482 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.856460 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.856798 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.856778 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:39.861438 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:39.861415 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzrqb\" (UniqueName: \"kubernetes.io/projected/00ab0fc2-d0ce-48c7-a450-955b22b0e9b5-kube-api-access-bzrqb\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-dln9j\" (UID: \"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:40.015740 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:40.015639 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:50:40.151129 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:40.151102 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j"] Jun 11 18:50:40.153387 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:50:40.153346 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00ab0fc2_d0ce_48c7_a450_955b22b0e9b5.slice/crio-e7c33787bed083864fa68a4dd9dd2e7768bb8bd09055daa247bd0540c5374639 WatchSource:0}: Error finding container e7c33787bed083864fa68a4dd9dd2e7768bb8bd09055daa247bd0540c5374639: Status 404 returned error can't find the container with id e7c33787bed083864fa68a4dd9dd2e7768bb8bd09055daa247bd0540c5374639 Jun 11 18:50:40.178838 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:40.178812 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" event={"ID":"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5","Type":"ContainerStarted","Data":"e7c33787bed083864fa68a4dd9dd2e7768bb8bd09055daa247bd0540c5374639"} Jun 11 18:50:41.184559 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:41.184522 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" event={"ID":"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5","Type":"ContainerStarted","Data":"e7425ccd543a49b1eaf6e0a93824af434ec05e18a07e5da09ef7c8d89cc0f7e5"} Jun 11 18:50:58.919728 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:58.919658 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 18:50:58.921548 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:58.921528 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 18:50:58.925421 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:58.925399 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 18:50:58.927449 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:50:58.927435 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 18:51:02.954700 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:02.954645 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5"] Jun 11 18:51:02.958507 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:02.958481 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:02.960778 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:02.960756 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 11 18:51:02.963255 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:02.963232 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86aa963a-1190-477c-ac13-3fec1f93da05-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:02.963395 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:02.963261 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86aa963a-1190-477c-ac13-3fec1f93da05-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:02.963395 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:02.963295 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86aa963a-1190-477c-ac13-3fec1f93da05-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:02.963395 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:02.963347 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58g49\" (UniqueName: \"kubernetes.io/projected/86aa963a-1190-477c-ac13-3fec1f93da05-kube-api-access-58g49\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:02.963395 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:02.963375 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86aa963a-1190-477c-ac13-3fec1f93da05-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:02.963627 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:02.963411 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/86aa963a-1190-477c-ac13-3fec1f93da05-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:02.968897 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:02.968872 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5"] Jun 11 18:51:03.064403 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:03.064367 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/86aa963a-1190-477c-ac13-3fec1f93da05-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:03.064403 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:03.064416 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86aa963a-1190-477c-ac13-3fec1f93da05-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:03.064657 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:03.064437 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86aa963a-1190-477c-ac13-3fec1f93da05-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:03.064657 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:03.064471 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86aa963a-1190-477c-ac13-3fec1f93da05-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:03.064657 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:03.064496 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58g49\" (UniqueName: \"kubernetes.io/projected/86aa963a-1190-477c-ac13-3fec1f93da05-kube-api-access-58g49\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:03.064657 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:03.064521 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86aa963a-1190-477c-ac13-3fec1f93da05-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:03.064861 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:03.064838 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86aa963a-1190-477c-ac13-3fec1f93da05-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:03.064939 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:03.064876 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/86aa963a-1190-477c-ac13-3fec1f93da05-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:03.064939 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:03.064903 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86aa963a-1190-477c-ac13-3fec1f93da05-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:03.066829 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:03.066804 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86aa963a-1190-477c-ac13-3fec1f93da05-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:03.067023 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:03.067004 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86aa963a-1190-477c-ac13-3fec1f93da05-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:03.072480 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:03.072456 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58g49\" (UniqueName: \"kubernetes.io/projected/86aa963a-1190-477c-ac13-3fec1f93da05-kube-api-access-58g49\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r9hl5\" (UID: \"86aa963a-1190-477c-ac13-3fec1f93da05\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:03.268753 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:03.268670 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:03.408010 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:03.407983 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5"] Jun 11 18:51:03.409428 ip-10-0-138-156 kubenswrapper[2561]: W0611 18:51:03.409400 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86aa963a_1190_477c_ac13_3fec1f93da05.slice/crio-316d28ddf244e6816b3658651808ed46f51704f1c7b8ff4957ecb6ff32090018 WatchSource:0}: Error finding container 316d28ddf244e6816b3658651808ed46f51704f1c7b8ff4957ecb6ff32090018: Status 404 returned error can't find the container with id 316d28ddf244e6816b3658651808ed46f51704f1c7b8ff4957ecb6ff32090018 Jun 11 18:51:04.299349 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:04.299311 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" event={"ID":"86aa963a-1190-477c-ac13-3fec1f93da05","Type":"ContainerStarted","Data":"6ecbf13cbf3e23cec8ea8a342443724767e7d8b7944c3afa8271a015ef8bfc50"} Jun 11 18:51:04.299839 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:04.299360 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" event={"ID":"86aa963a-1190-477c-ac13-3fec1f93da05","Type":"ContainerStarted","Data":"316d28ddf244e6816b3658651808ed46f51704f1c7b8ff4957ecb6ff32090018"} Jun 11 18:51:05.305757 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:05.305722 2561 generic.go:356] "Generic (PLEG): container finished" podID="86aa963a-1190-477c-ac13-3fec1f93da05" containerID="6ecbf13cbf3e23cec8ea8a342443724767e7d8b7944c3afa8271a015ef8bfc50" exitCode=0 Jun 11 18:51:05.306250 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:05.305769 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" event={"ID":"86aa963a-1190-477c-ac13-3fec1f93da05","Type":"ContainerDied","Data":"6ecbf13cbf3e23cec8ea8a342443724767e7d8b7944c3afa8271a015ef8bfc50"} Jun 11 18:51:07.319138 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:07.319099 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" event={"ID":"86aa963a-1190-477c-ac13-3fec1f93da05","Type":"ContainerStarted","Data":"c78787c0a67d30f31f09ef121c266408a69355afdc269a8a0482accf8a07db97"} Jun 11 18:51:07.320499 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:07.320476 2561 generic.go:356] "Generic (PLEG): container finished" podID="0caea8f3-61a6-480b-8885-4ec8561b6f5d" containerID="82b969541b3639c63bc8b4a33751d7c567713be96825e664bcd82df5f5e580f7" exitCode=0 Jun 11 18:51:07.320634 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:07.320555 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" event={"ID":"0caea8f3-61a6-480b-8885-4ec8561b6f5d","Type":"ContainerDied","Data":"82b969541b3639c63bc8b4a33751d7c567713be96825e664bcd82df5f5e580f7"} Jun 11 18:51:07.337776 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:07.337726 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" podStartSLOduration=2.403723695 podStartE2EDuration="5.337710329s" totalImagesPullingTime="1.258834429s" totalInitContainerRuntime="1.675152205s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:51:07.335858643 +0000 UTC m=+908.903932124" watchObservedRunningTime="2026-06-11 18:51:07.337710329 +0000 UTC m=+908.905783809" Jun 11 18:51:08.327240 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:08.327191 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" event={"ID":"0caea8f3-61a6-480b-8885-4ec8561b6f5d","Type":"ContainerStarted","Data":"6b486d1c694ea87dbcba86209c45808203c5337a63f5d5fcefa0999bf2f6a1fb"} Jun 11 18:51:08.345318 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:08.345269 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" podStartSLOduration=2.151579293 podStartE2EDuration="1m1.345253058s" totalImagesPullingTime="5.365852841s" totalInitContainerRuntime="53.827820924s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:50:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:51:08.344107291 +0000 UTC m=+909.912180764" watchObservedRunningTime="2026-06-11 18:51:08.345253058 +0000 UTC m=+909.913326539" Jun 11 18:51:13.269160 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:13.269118 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:13.269160 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:13.269168 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:13.282772 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:13.282743 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:13.363497 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:13.363468 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r9hl5" Jun 11 18:51:17.645043 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:17.645004 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:51:17.645540 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:17.645358 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:51:17.657628 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:17.657604 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:51:18.392422 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:18.392387 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-58sj7" Jun 11 18:51:34.454158 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:34.454120 2561 generic.go:356] "Generic (PLEG): container finished" podID="00ab0fc2-d0ce-48c7-a450-955b22b0e9b5" containerID="e7425ccd543a49b1eaf6e0a93824af434ec05e18a07e5da09ef7c8d89cc0f7e5" exitCode=0 Jun 11 18:51:34.454595 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:34.454194 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" event={"ID":"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5","Type":"ContainerDied","Data":"e7425ccd543a49b1eaf6e0a93824af434ec05e18a07e5da09ef7c8d89cc0f7e5"} Jun 11 18:51:35.460683 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:35.460651 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" event={"ID":"00ab0fc2-d0ce-48c7-a450-955b22b0e9b5","Type":"ContainerStarted","Data":"715653bb89736ee948d8fbed24a00b0fe8844214c2bca263d5aacf4272f001b1"} Jun 11 18:51:35.483361 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:35.483301 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" podStartSLOduration=2.173064278 podStartE2EDuration="56.483285446s" totalImagesPullingTime="199.007838ms" totalInitContainerRuntime="54.11121333s" isStatefulPod=false podCreationTimestamp="2026-06-11 18:50:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 18:51:35.481410347 +0000 UTC m=+937.049483831" watchObservedRunningTime="2026-06-11 18:51:35.483285446 +0000 UTC m=+937.051358925" Jun 11 18:51:40.016353 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:40.016309 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:51:40.016943 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:40.016391 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:51:40.028903 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:40.028879 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:51:40.494992 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:51:40.494961 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-dln9j" Jun 11 18:55:58.955833 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:55:58.955800 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 18:55:58.958314 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:55:58.958288 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 18:55:58.962313 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:55:58.962291 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 18:55:58.964522 ip-10-0-138-156 kubenswrapper[2561]: I0611 18:55:58.964507 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 19:00:58.984934 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:00:58.984903 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 19:00:58.988399 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:00:58.988372 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 19:00:58.990918 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:00:58.990896 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 19:00:58.994327 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:00:58.994308 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 19:05:59.021278 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:05:59.021245 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 19:05:59.025849 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:05:59.025825 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 19:05:59.027440 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:05:59.027417 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 19:05:59.031581 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:05:59.031564 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 19:10:59.053642 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:10:59.053612 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 19:10:59.057259 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:10:59.057234 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 19:10:59.059600 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:10:59.059581 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 19:10:59.063142 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:10:59.063127 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 19:13:36.280657 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.280620 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6"] Jun 11 19:13:36.284643 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.284620 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.292811 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.292782 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6"] Jun 11 19:13:36.443821 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.443780 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-ca\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.443821 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.443826 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-hub\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.444039 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.443846 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.444039 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.443960 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.444039 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.443995 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/caf26de4-38b2-446a-8b66-81b625a6bbeb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.444137 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.444116 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz847\" (UniqueName: \"kubernetes.io/projected/caf26de4-38b2-446a-8b66-81b625a6bbeb-kube-api-access-jz847\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.545183 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.545083 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.545183 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.545121 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/caf26de4-38b2-446a-8b66-81b625a6bbeb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.545410 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.545191 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz847\" (UniqueName: \"kubernetes.io/projected/caf26de4-38b2-446a-8b66-81b625a6bbeb-kube-api-access-jz847\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.545410 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.545209 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-ca\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.545410 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.545228 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-hub\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.545410 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.545247 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.546019 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.545986 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/caf26de4-38b2-446a-8b66-81b625a6bbeb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.547927 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.547899 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.548457 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.548426 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-ca\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.548552 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.548534 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.548552 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.548542 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-hub\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.552107 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.552087 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz847\" (UniqueName: \"kubernetes.io/projected/caf26de4-38b2-446a-8b66-81b625a6bbeb-kube-api-access-jz847\") pod \"cluster-proxy-proxy-agent-59cf4b445b-nrbz6\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.594120 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.594079 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:13:36.734418 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.734389 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6"] Jun 11 19:13:36.736396 ip-10-0-138-156 kubenswrapper[2561]: W0611 19:13:36.736357 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcaf26de4_38b2_446a_8b66_81b625a6bbeb.slice/crio-59a19ea1b52455bf681fee8adf9ad0b19087271e02047921f9ed649704d311db WatchSource:0}: Error finding container 59a19ea1b52455bf681fee8adf9ad0b19087271e02047921f9ed649704d311db: Status 404 returned error can't find the container with id 59a19ea1b52455bf681fee8adf9ad0b19087271e02047921f9ed649704d311db Jun 11 19:13:36.738285 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.738259 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 19:13:36.859527 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.859486 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" event={"ID":"caf26de4-38b2-446a-8b66-81b625a6bbeb","Type":"ContainerStarted","Data":"59a19ea1b52455bf681fee8adf9ad0b19087271e02047921f9ed649704d311db"} Jun 11 19:13:36.955533 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.955497 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8"] Jun 11 19:13:36.955712 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:36.955667 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" podUID="21ff8dcc-639e-4865-af63-8f609ed3544e" containerName="addon-agent" containerID="cri-o://9f9028c54ea6d0b58add36f794339399d283a67fab7b89a86cc719cb4e012262" gracePeriod=30 Jun 11 19:13:37.200429 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.200403 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" Jun 11 19:13:37.354213 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.354160 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtjj8\" (UniqueName: \"kubernetes.io/projected/21ff8dcc-639e-4865-af63-8f609ed3544e-kube-api-access-gtjj8\") pod \"21ff8dcc-639e-4865-af63-8f609ed3544e\" (UID: \"21ff8dcc-639e-4865-af63-8f609ed3544e\") " Jun 11 19:13:37.354579 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.354264 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/21ff8dcc-639e-4865-af63-8f609ed3544e-hub-kubeconfig\") pod \"21ff8dcc-639e-4865-af63-8f609ed3544e\" (UID: \"21ff8dcc-639e-4865-af63-8f609ed3544e\") " Jun 11 19:13:37.356344 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.356310 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21ff8dcc-639e-4865-af63-8f609ed3544e-kube-api-access-gtjj8" (OuterVolumeSpecName: "kube-api-access-gtjj8") pod "21ff8dcc-639e-4865-af63-8f609ed3544e" (UID: "21ff8dcc-639e-4865-af63-8f609ed3544e"). InnerVolumeSpecName "kube-api-access-gtjj8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 19:13:37.356436 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.356348 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21ff8dcc-639e-4865-af63-8f609ed3544e-hub-kubeconfig" (OuterVolumeSpecName: "hub-kubeconfig") pod "21ff8dcc-639e-4865-af63-8f609ed3544e" (UID: "21ff8dcc-639e-4865-af63-8f609ed3544e"). InnerVolumeSpecName "hub-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 19:13:37.455587 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.455493 2561 reconciler_common.go:299] "Volume detached for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/21ff8dcc-639e-4865-af63-8f609ed3544e-hub-kubeconfig\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:13:37.455587 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.455530 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtjj8\" (UniqueName: \"kubernetes.io/projected/21ff8dcc-639e-4865-af63-8f609ed3544e-kube-api-access-gtjj8\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:13:37.867918 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.867841 2561 generic.go:356] "Generic (PLEG): container finished" podID="21ff8dcc-639e-4865-af63-8f609ed3544e" containerID="9f9028c54ea6d0b58add36f794339399d283a67fab7b89a86cc719cb4e012262" exitCode=0 Jun 11 19:13:37.867918 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.867900 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" Jun 11 19:13:37.868138 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.867924 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" event={"ID":"21ff8dcc-639e-4865-af63-8f609ed3544e","Type":"ContainerDied","Data":"9f9028c54ea6d0b58add36f794339399d283a67fab7b89a86cc719cb4e012262"} Jun 11 19:13:37.868138 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.867977 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8" event={"ID":"21ff8dcc-639e-4865-af63-8f609ed3544e","Type":"ContainerDied","Data":"d6e6db26624b1cc351403fb20c744c62395ab98e448728f50b9e7592013ea033"} Jun 11 19:13:37.868138 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.868003 2561 scope.go:117] "RemoveContainer" containerID="9f9028c54ea6d0b58add36f794339399d283a67fab7b89a86cc719cb4e012262" Jun 11 19:13:37.880943 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.880921 2561 scope.go:117] "RemoveContainer" containerID="9f9028c54ea6d0b58add36f794339399d283a67fab7b89a86cc719cb4e012262" Jun 11 19:13:37.881320 ip-10-0-138-156 kubenswrapper[2561]: E0611 19:13:37.881274 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f9028c54ea6d0b58add36f794339399d283a67fab7b89a86cc719cb4e012262\": container with ID starting with 9f9028c54ea6d0b58add36f794339399d283a67fab7b89a86cc719cb4e012262 not found: ID does not exist" containerID="9f9028c54ea6d0b58add36f794339399d283a67fab7b89a86cc719cb4e012262" Jun 11 19:13:37.881520 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.881313 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f9028c54ea6d0b58add36f794339399d283a67fab7b89a86cc719cb4e012262"} err="failed to get container status \"9f9028c54ea6d0b58add36f794339399d283a67fab7b89a86cc719cb4e012262\": rpc error: code = NotFound desc = could not find container \"9f9028c54ea6d0b58add36f794339399d283a67fab7b89a86cc719cb4e012262\": container with ID starting with 9f9028c54ea6d0b58add36f794339399d283a67fab7b89a86cc719cb4e012262 not found: ID does not exist" Jun 11 19:13:37.894849 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.894807 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8"] Jun 11 19:13:37.898380 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:37.898349 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-667dd5cfb8-ff2g8"] Jun 11 19:13:38.918571 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:38.918537 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21ff8dcc-639e-4865-af63-8f609ed3544e" path="/var/lib/kubelet/pods/21ff8dcc-639e-4865-af63-8f609ed3544e/volumes" Jun 11 19:13:41.893245 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:41.893207 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" event={"ID":"caf26de4-38b2-446a-8b66-81b625a6bbeb","Type":"ContainerStarted","Data":"0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad"} Jun 11 19:13:41.893767 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:41.893259 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" event={"ID":"caf26de4-38b2-446a-8b66-81b625a6bbeb","Type":"ContainerStarted","Data":"f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f"} Jun 11 19:13:41.893767 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:41.893278 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" event={"ID":"caf26de4-38b2-446a-8b66-81b625a6bbeb","Type":"ContainerStarted","Data":"fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3"} Jun 11 19:13:41.911452 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:41.911395 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" podStartSLOduration=1.32428686 podStartE2EDuration="5.911381036s" totalImagesPullingTime="4.587094176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 19:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 19:13:41.910283266 +0000 UTC m=+2263.478356739" watchObservedRunningTime="2026-06-11 19:13:41.911381036 +0000 UTC m=+2263.479454515" Jun 11 19:13:41.936678 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:41.936645 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx"] Jun 11 19:13:41.936863 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:41.936826 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" podUID="7805e125-ba9b-4713-ac48-3678fba0895d" containerName="proxy-agent" containerID="cri-o://a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec" gracePeriod=30 Jun 11 19:13:41.936998 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:41.936853 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" podUID="7805e125-ba9b-4713-ac48-3678fba0895d" containerName="service-proxy" containerID="cri-o://d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f" gracePeriod=30 Jun 11 19:13:41.936998 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:41.936876 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" podUID="7805e125-ba9b-4713-ac48-3678fba0895d" containerName="addon-agent" containerID="cri-o://497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02" gracePeriod=30 Jun 11 19:13:42.186940 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.186913 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 19:13:42.299739 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.299699 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-ca\") pod \"7805e125-ba9b-4713-ac48-3678fba0895d\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " Jun 11 19:13:42.299939 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.299751 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-service-proxy-server-cert\") pod \"7805e125-ba9b-4713-ac48-3678fba0895d\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " Jun 11 19:13:42.299939 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.299849 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7805e125-ba9b-4713-ac48-3678fba0895d-additional-service-ca\") pod \"7805e125-ba9b-4713-ac48-3678fba0895d\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " Jun 11 19:13:42.299939 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.299901 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-hub\") pod \"7805e125-ba9b-4713-ac48-3678fba0895d\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " Jun 11 19:13:42.299939 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.299925 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-hub-kubeconfig\") pod \"7805e125-ba9b-4713-ac48-3678fba0895d\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " Jun 11 19:13:42.300157 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.299955 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn5sm\" (UniqueName: \"kubernetes.io/projected/7805e125-ba9b-4713-ac48-3678fba0895d-kube-api-access-gn5sm\") pod \"7805e125-ba9b-4713-ac48-3678fba0895d\" (UID: \"7805e125-ba9b-4713-ac48-3678fba0895d\") " Jun 11 19:13:42.300321 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.300289 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7805e125-ba9b-4713-ac48-3678fba0895d-additional-service-ca" (OuterVolumeSpecName: "additional-service-ca") pod "7805e125-ba9b-4713-ac48-3678fba0895d" (UID: "7805e125-ba9b-4713-ac48-3678fba0895d"). InnerVolumeSpecName "additional-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 19:13:42.301972 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.301946 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-ca" (OuterVolumeSpecName: "ca") pod "7805e125-ba9b-4713-ac48-3678fba0895d" (UID: "7805e125-ba9b-4713-ac48-3678fba0895d"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 19:13:42.302081 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.302053 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7805e125-ba9b-4713-ac48-3678fba0895d-kube-api-access-gn5sm" (OuterVolumeSpecName: "kube-api-access-gn5sm") pod "7805e125-ba9b-4713-ac48-3678fba0895d" (UID: "7805e125-ba9b-4713-ac48-3678fba0895d"). InnerVolumeSpecName "kube-api-access-gn5sm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 19:13:42.302127 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.302079 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-hub" (OuterVolumeSpecName: "hub") pod "7805e125-ba9b-4713-ac48-3678fba0895d" (UID: "7805e125-ba9b-4713-ac48-3678fba0895d"). InnerVolumeSpecName "hub". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 19:13:42.302376 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.302351 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-hub-kubeconfig" (OuterVolumeSpecName: "hub-kubeconfig") pod "7805e125-ba9b-4713-ac48-3678fba0895d" (UID: "7805e125-ba9b-4713-ac48-3678fba0895d"). InnerVolumeSpecName "hub-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 19:13:42.302460 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.302378 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-service-proxy-server-cert" (OuterVolumeSpecName: "service-proxy-server-cert") pod "7805e125-ba9b-4713-ac48-3678fba0895d" (UID: "7805e125-ba9b-4713-ac48-3678fba0895d"). InnerVolumeSpecName "service-proxy-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 19:13:42.401445 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.401414 2561 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-ca\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:13:42.401445 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.401441 2561 reconciler_common.go:299] "Volume detached for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-service-proxy-server-cert\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:13:42.401445 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.401451 2561 reconciler_common.go:299] "Volume detached for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7805e125-ba9b-4713-ac48-3678fba0895d-additional-service-ca\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:13:42.401665 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.401460 2561 reconciler_common.go:299] "Volume detached for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-hub\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:13:42.401665 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.401469 2561 reconciler_common.go:299] "Volume detached for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7805e125-ba9b-4713-ac48-3678fba0895d-hub-kubeconfig\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:13:42.401665 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.401478 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn5sm\" (UniqueName: \"kubernetes.io/projected/7805e125-ba9b-4713-ac48-3678fba0895d-kube-api-access-gn5sm\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:13:42.899553 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.899515 2561 generic.go:356] "Generic (PLEG): container finished" podID="7805e125-ba9b-4713-ac48-3678fba0895d" containerID="d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f" exitCode=2 Jun 11 19:13:42.899553 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.899551 2561 generic.go:356] "Generic (PLEG): container finished" podID="7805e125-ba9b-4713-ac48-3678fba0895d" containerID="497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02" exitCode=2 Jun 11 19:13:42.899553 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.899560 2561 generic.go:356] "Generic (PLEG): container finished" podID="7805e125-ba9b-4713-ac48-3678fba0895d" containerID="a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec" exitCode=2 Jun 11 19:13:42.900108 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.899573 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" Jun 11 19:13:42.900108 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.899589 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" event={"ID":"7805e125-ba9b-4713-ac48-3678fba0895d","Type":"ContainerDied","Data":"d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f"} Jun 11 19:13:42.900108 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.899635 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" event={"ID":"7805e125-ba9b-4713-ac48-3678fba0895d","Type":"ContainerDied","Data":"497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02"} Jun 11 19:13:42.900108 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.899648 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" event={"ID":"7805e125-ba9b-4713-ac48-3678fba0895d","Type":"ContainerDied","Data":"a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec"} Jun 11 19:13:42.900108 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.899661 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx" event={"ID":"7805e125-ba9b-4713-ac48-3678fba0895d","Type":"ContainerDied","Data":"21fb3a322707058a70c0a9020b42fbb85b0210a065a09e717754ae3947db6a9b"} Jun 11 19:13:42.900108 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.899678 2561 scope.go:117] "RemoveContainer" containerID="d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f" Jun 11 19:13:42.909091 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.909069 2561 scope.go:117] "RemoveContainer" containerID="497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02" Jun 11 19:13:42.917777 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.917759 2561 scope.go:117] "RemoveContainer" containerID="a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec" Jun 11 19:13:42.921483 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.921460 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx"] Jun 11 19:13:42.925260 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.925236 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-779bf9c455-ldghx"] Jun 11 19:13:42.926192 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.926151 2561 scope.go:117] "RemoveContainer" containerID="d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f" Jun 11 19:13:42.926445 ip-10-0-138-156 kubenswrapper[2561]: E0611 19:13:42.926420 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f\": container with ID starting with d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f not found: ID does not exist" containerID="d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f" Jun 11 19:13:42.926500 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.926451 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f"} err="failed to get container status \"d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f\": rpc error: code = NotFound desc = could not find container \"d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f\": container with ID starting with d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f not found: ID does not exist" Jun 11 19:13:42.926500 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.926470 2561 scope.go:117] "RemoveContainer" containerID="497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02" Jun 11 19:13:42.926731 ip-10-0-138-156 kubenswrapper[2561]: E0611 19:13:42.926711 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02\": container with ID starting with 497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02 not found: ID does not exist" containerID="497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02" Jun 11 19:13:42.926770 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.926738 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02"} err="failed to get container status \"497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02\": rpc error: code = NotFound desc = could not find container \"497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02\": container with ID starting with 497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02 not found: ID does not exist" Jun 11 19:13:42.926770 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.926754 2561 scope.go:117] "RemoveContainer" containerID="a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec" Jun 11 19:13:42.926956 ip-10-0-138-156 kubenswrapper[2561]: E0611 19:13:42.926939 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec\": container with ID starting with a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec not found: ID does not exist" containerID="a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec" Jun 11 19:13:42.927003 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.926961 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec"} err="failed to get container status \"a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec\": rpc error: code = NotFound desc = could not find container \"a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec\": container with ID starting with a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec not found: ID does not exist" Jun 11 19:13:42.927003 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.926974 2561 scope.go:117] "RemoveContainer" containerID="d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f" Jun 11 19:13:42.927232 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.927209 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f"} err="failed to get container status \"d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f\": rpc error: code = NotFound desc = could not find container \"d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f\": container with ID starting with d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f not found: ID does not exist" Jun 11 19:13:42.927322 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.927234 2561 scope.go:117] "RemoveContainer" containerID="497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02" Jun 11 19:13:42.927521 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.927503 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02"} err="failed to get container status \"497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02\": rpc error: code = NotFound desc = could not find container \"497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02\": container with ID starting with 497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02 not found: ID does not exist" Jun 11 19:13:42.927565 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.927522 2561 scope.go:117] "RemoveContainer" containerID="a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec" Jun 11 19:13:42.927776 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.927754 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec"} err="failed to get container status \"a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec\": rpc error: code = NotFound desc = could not find container \"a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec\": container with ID starting with a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec not found: ID does not exist" Jun 11 19:13:42.927846 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.927778 2561 scope.go:117] "RemoveContainer" containerID="d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f" Jun 11 19:13:42.927990 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.927973 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f"} err="failed to get container status \"d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f\": rpc error: code = NotFound desc = could not find container \"d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f\": container with ID starting with d3881710a97befe8b4eb51dca0e3d52432432114add2b0346f9beeca0eed433f not found: ID does not exist" Jun 11 19:13:42.928036 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.927991 2561 scope.go:117] "RemoveContainer" containerID="497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02" Jun 11 19:13:42.928205 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.928185 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02"} err="failed to get container status \"497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02\": rpc error: code = NotFound desc = could not find container \"497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02\": container with ID starting with 497fc32d29b54f480fd4187be4d835dcbf8bb96d266a0f7e05acdba464f31c02 not found: ID does not exist" Jun 11 19:13:42.928282 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.928209 2561 scope.go:117] "RemoveContainer" containerID="a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec" Jun 11 19:13:42.928439 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:42.928421 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec"} err="failed to get container status \"a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec\": rpc error: code = NotFound desc = could not find container \"a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec\": container with ID starting with a007c1615ac5add487c2657e04d5aed5a61a070ff74d589188fd78b15e64d6ec not found: ID does not exist" Jun 11 19:13:44.919596 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:13:44.919564 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7805e125-ba9b-4713-ac48-3678fba0895d" path="/var/lib/kubelet/pods/7805e125-ba9b-4713-ac48-3678fba0895d/volumes" Jun 11 19:14:03.098624 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.098586 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6"] Jun 11 19:14:03.099084 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.098804 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" podUID="caf26de4-38b2-446a-8b66-81b625a6bbeb" containerName="proxy-agent" containerID="cri-o://fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3" gracePeriod=30 Jun 11 19:14:03.099084 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.098850 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" podUID="caf26de4-38b2-446a-8b66-81b625a6bbeb" containerName="service-proxy" containerID="cri-o://0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad" gracePeriod=30 Jun 11 19:14:03.099084 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.098850 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" podUID="caf26de4-38b2-446a-8b66-81b625a6bbeb" containerName="addon-agent" containerID="cri-o://f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f" gracePeriod=30 Jun 11 19:14:03.347515 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.347490 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:14:03.388795 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.388715 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-service-proxy-server-cert\") pod \"caf26de4-38b2-446a-8b66-81b625a6bbeb\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " Jun 11 19:14:03.388795 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.388752 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/caf26de4-38b2-446a-8b66-81b625a6bbeb-additional-service-ca\") pod \"caf26de4-38b2-446a-8b66-81b625a6bbeb\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " Jun 11 19:14:03.388795 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.388776 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-ca\") pod \"caf26de4-38b2-446a-8b66-81b625a6bbeb\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " Jun 11 19:14:03.389065 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.388843 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-hub-kubeconfig\") pod \"caf26de4-38b2-446a-8b66-81b625a6bbeb\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " Jun 11 19:14:03.389065 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.388900 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-hub\") pod \"caf26de4-38b2-446a-8b66-81b625a6bbeb\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " Jun 11 19:14:03.389065 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.388955 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jz847\" (UniqueName: \"kubernetes.io/projected/caf26de4-38b2-446a-8b66-81b625a6bbeb-kube-api-access-jz847\") pod \"caf26de4-38b2-446a-8b66-81b625a6bbeb\" (UID: \"caf26de4-38b2-446a-8b66-81b625a6bbeb\") " Jun 11 19:14:03.389229 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.389134 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caf26de4-38b2-446a-8b66-81b625a6bbeb-additional-service-ca" (OuterVolumeSpecName: "additional-service-ca") pod "caf26de4-38b2-446a-8b66-81b625a6bbeb" (UID: "caf26de4-38b2-446a-8b66-81b625a6bbeb"). InnerVolumeSpecName "additional-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 19:14:03.389346 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.389322 2561 reconciler_common.go:299] "Volume detached for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/caf26de4-38b2-446a-8b66-81b625a6bbeb-additional-service-ca\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:14:03.391457 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.391407 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-hub" (OuterVolumeSpecName: "hub") pod "caf26de4-38b2-446a-8b66-81b625a6bbeb" (UID: "caf26de4-38b2-446a-8b66-81b625a6bbeb"). InnerVolumeSpecName "hub". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 19:14:03.391565 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.391465 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-hub-kubeconfig" (OuterVolumeSpecName: "hub-kubeconfig") pod "caf26de4-38b2-446a-8b66-81b625a6bbeb" (UID: "caf26de4-38b2-446a-8b66-81b625a6bbeb"). InnerVolumeSpecName "hub-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 19:14:03.391565 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.391487 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-ca" (OuterVolumeSpecName: "ca") pod "caf26de4-38b2-446a-8b66-81b625a6bbeb" (UID: "caf26de4-38b2-446a-8b66-81b625a6bbeb"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 19:14:03.391565 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.391498 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-service-proxy-server-cert" (OuterVolumeSpecName: "service-proxy-server-cert") pod "caf26de4-38b2-446a-8b66-81b625a6bbeb" (UID: "caf26de4-38b2-446a-8b66-81b625a6bbeb"). InnerVolumeSpecName "service-proxy-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 19:14:03.391744 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.391607 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf26de4-38b2-446a-8b66-81b625a6bbeb-kube-api-access-jz847" (OuterVolumeSpecName: "kube-api-access-jz847") pod "caf26de4-38b2-446a-8b66-81b625a6bbeb" (UID: "caf26de4-38b2-446a-8b66-81b625a6bbeb"). InnerVolumeSpecName "kube-api-access-jz847". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 19:14:03.490067 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.490028 2561 reconciler_common.go:299] "Volume detached for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-service-proxy-server-cert\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:14:03.490067 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.490060 2561 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-ca\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:14:03.490067 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.490069 2561 reconciler_common.go:299] "Volume detached for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-hub-kubeconfig\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:14:03.490067 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.490079 2561 reconciler_common.go:299] "Volume detached for volume \"hub\" (UniqueName: \"kubernetes.io/secret/caf26de4-38b2-446a-8b66-81b625a6bbeb-hub\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:14:03.490358 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:03.490087 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz847\" (UniqueName: \"kubernetes.io/projected/caf26de4-38b2-446a-8b66-81b625a6bbeb-kube-api-access-jz847\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:14:04.015813 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.015772 2561 generic.go:356] "Generic (PLEG): container finished" podID="caf26de4-38b2-446a-8b66-81b625a6bbeb" containerID="0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad" exitCode=2 Jun 11 19:14:04.015813 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.015801 2561 generic.go:356] "Generic (PLEG): container finished" podID="caf26de4-38b2-446a-8b66-81b625a6bbeb" containerID="f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f" exitCode=2 Jun 11 19:14:04.015813 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.015811 2561 generic.go:356] "Generic (PLEG): container finished" podID="caf26de4-38b2-446a-8b66-81b625a6bbeb" containerID="fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3" exitCode=2 Jun 11 19:14:04.016083 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.015824 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" Jun 11 19:14:04.016083 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.015851 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" event={"ID":"caf26de4-38b2-446a-8b66-81b625a6bbeb","Type":"ContainerDied","Data":"0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad"} Jun 11 19:14:04.016083 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.015910 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" event={"ID":"caf26de4-38b2-446a-8b66-81b625a6bbeb","Type":"ContainerDied","Data":"f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f"} Jun 11 19:14:04.016083 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.015924 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" event={"ID":"caf26de4-38b2-446a-8b66-81b625a6bbeb","Type":"ContainerDied","Data":"fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3"} Jun 11 19:14:04.016083 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.015936 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6" event={"ID":"caf26de4-38b2-446a-8b66-81b625a6bbeb","Type":"ContainerDied","Data":"59a19ea1b52455bf681fee8adf9ad0b19087271e02047921f9ed649704d311db"} Jun 11 19:14:04.016083 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.015954 2561 scope.go:117] "RemoveContainer" containerID="0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad" Jun 11 19:14:04.029400 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.029379 2561 scope.go:117] "RemoveContainer" containerID="f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f" Jun 11 19:14:04.037300 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.037281 2561 scope.go:117] "RemoveContainer" containerID="fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3" Jun 11 19:14:04.041667 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.041643 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6"] Jun 11 19:14:04.045021 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.045004 2561 scope.go:117] "RemoveContainer" containerID="0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad" Jun 11 19:14:04.045464 ip-10-0-138-156 kubenswrapper[2561]: E0611 19:14:04.045426 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad\": container with ID starting with 0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad not found: ID does not exist" containerID="0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad" Jun 11 19:14:04.045566 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.045474 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad"} err="failed to get container status \"0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad\": rpc error: code = NotFound desc = could not find container \"0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad\": container with ID starting with 0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad not found: ID does not exist" Jun 11 19:14:04.045566 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.045505 2561 scope.go:117] "RemoveContainer" containerID="f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f" Jun 11 19:14:04.045833 ip-10-0-138-156 kubenswrapper[2561]: E0611 19:14:04.045806 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f\": container with ID starting with f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f not found: ID does not exist" containerID="f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f" Jun 11 19:14:04.045959 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.045839 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f"} err="failed to get container status \"f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f\": rpc error: code = NotFound desc = could not find container \"f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f\": container with ID starting with f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f not found: ID does not exist" Jun 11 19:14:04.045959 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.045859 2561 scope.go:117] "RemoveContainer" containerID="fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3" Jun 11 19:14:04.048476 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.048454 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59cf4b445b-nrbz6"] Jun 11 19:14:04.049089 ip-10-0-138-156 kubenswrapper[2561]: E0611 19:14:04.049065 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3\": container with ID starting with fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3 not found: ID does not exist" containerID="fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3" Jun 11 19:14:04.049238 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.049091 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3"} err="failed to get container status \"fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3\": rpc error: code = NotFound desc = could not find container \"fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3\": container with ID starting with fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3 not found: ID does not exist" Jun 11 19:14:04.049238 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.049108 2561 scope.go:117] "RemoveContainer" containerID="0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad" Jun 11 19:14:04.049616 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.049599 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad"} err="failed to get container status \"0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad\": rpc error: code = NotFound desc = could not find container \"0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad\": container with ID starting with 0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad not found: ID does not exist" Jun 11 19:14:04.049616 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.049617 2561 scope.go:117] "RemoveContainer" containerID="f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f" Jun 11 19:14:04.049867 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.049846 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f"} err="failed to get container status \"f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f\": rpc error: code = NotFound desc = could not find container \"f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f\": container with ID starting with f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f not found: ID does not exist" Jun 11 19:14:04.049867 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.049870 2561 scope.go:117] "RemoveContainer" containerID="fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3" Jun 11 19:14:04.050094 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.050078 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3"} err="failed to get container status \"fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3\": rpc error: code = NotFound desc = could not find container \"fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3\": container with ID starting with fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3 not found: ID does not exist" Jun 11 19:14:04.050094 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.050095 2561 scope.go:117] "RemoveContainer" containerID="0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad" Jun 11 19:14:04.050334 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.050317 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad"} err="failed to get container status \"0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad\": rpc error: code = NotFound desc = could not find container \"0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad\": container with ID starting with 0f6bf5e7341f49a7265922924b6e1edae9cc2ceb879b6a52fe1c5b7e637674ad not found: ID does not exist" Jun 11 19:14:04.050379 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.050335 2561 scope.go:117] "RemoveContainer" containerID="f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f" Jun 11 19:14:04.050542 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.050525 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f"} err="failed to get container status \"f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f\": rpc error: code = NotFound desc = could not find container \"f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f\": container with ID starting with f0f839f9bb2aebd0af8447f014287fa022e2f42df326737a8ca7cb698c7a8e2f not found: ID does not exist" Jun 11 19:14:04.050542 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.050541 2561 scope.go:117] "RemoveContainer" containerID="fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3" Jun 11 19:14:04.050757 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.050742 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3"} err="failed to get container status \"fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3\": rpc error: code = NotFound desc = could not find container \"fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3\": container with ID starting with fa99c2a7e43a6376ad628eee8a07bce116847674ce8fd1ce3460127bc06af0b3 not found: ID does not exist" Jun 11 19:14:04.918464 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:04.918433 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caf26de4-38b2-446a-8b66-81b625a6bbeb" path="/var/lib/kubelet/pods/caf26de4-38b2-446a-8b66-81b625a6bbeb/volumes" Jun 11 19:14:34.901877 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.901844 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g"] Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902263 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7805e125-ba9b-4713-ac48-3678fba0895d" containerName="service-proxy" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902278 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7805e125-ba9b-4713-ac48-3678fba0895d" containerName="service-proxy" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902293 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caf26de4-38b2-446a-8b66-81b625a6bbeb" containerName="proxy-agent" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902300 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="caf26de4-38b2-446a-8b66-81b625a6bbeb" containerName="proxy-agent" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902308 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7805e125-ba9b-4713-ac48-3678fba0895d" containerName="addon-agent" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902313 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7805e125-ba9b-4713-ac48-3678fba0895d" containerName="addon-agent" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902324 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21ff8dcc-639e-4865-af63-8f609ed3544e" containerName="addon-agent" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902331 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="21ff8dcc-639e-4865-af63-8f609ed3544e" containerName="addon-agent" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902345 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caf26de4-38b2-446a-8b66-81b625a6bbeb" containerName="service-proxy" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902351 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="caf26de4-38b2-446a-8b66-81b625a6bbeb" containerName="service-proxy" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902361 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7805e125-ba9b-4713-ac48-3678fba0895d" containerName="proxy-agent" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902366 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7805e125-ba9b-4713-ac48-3678fba0895d" containerName="proxy-agent" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902374 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caf26de4-38b2-446a-8b66-81b625a6bbeb" containerName="addon-agent" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902379 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="caf26de4-38b2-446a-8b66-81b625a6bbeb" containerName="addon-agent" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902443 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="caf26de4-38b2-446a-8b66-81b625a6bbeb" containerName="addon-agent" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902457 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7805e125-ba9b-4713-ac48-3678fba0895d" containerName="proxy-agent" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902464 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="21ff8dcc-639e-4865-af63-8f609ed3544e" containerName="addon-agent" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902470 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="caf26de4-38b2-446a-8b66-81b625a6bbeb" containerName="service-proxy" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902475 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7805e125-ba9b-4713-ac48-3678fba0895d" containerName="addon-agent" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902481 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7805e125-ba9b-4713-ac48-3678fba0895d" containerName="service-proxy" Jun 11 19:14:34.904370 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.902492 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="caf26de4-38b2-446a-8b66-81b625a6bbeb" containerName="proxy-agent" Jun 11 19:14:34.905753 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.905733 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:34.908156 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.908136 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3cbd9f4b-openshift-default-dockercfg-rqtsd\"" Jun 11 19:14:34.921476 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:34.921449 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g"] Jun 11 19:14:35.067735 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.067703 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-workload-certs\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.067947 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.067784 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79m9l\" (UniqueName: \"kubernetes.io/projected/a10e798a-985e-4ed6-acbe-ec17e9efe53f-kube-api-access-79m9l\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.067947 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.067818 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-token\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.067947 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.067882 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-envoy\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.067947 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.067907 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-data\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.067947 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.067930 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-podinfo\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.068236 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.068212 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istiod-ca-cert\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.068347 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.068327 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-credential-socket\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.068406 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.068393 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-workload-socket\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.169923 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.169829 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79m9l\" (UniqueName: \"kubernetes.io/projected/a10e798a-985e-4ed6-acbe-ec17e9efe53f-kube-api-access-79m9l\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.169923 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.169868 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-token\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.169923 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.169911 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-envoy\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.170283 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.169929 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-data\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.170283 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.169946 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-podinfo\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.170283 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.169974 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istiod-ca-cert\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.170283 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.170060 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-credential-socket\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.170283 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.170107 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-workload-socket\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.170283 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.170149 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-workload-certs\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.170649 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.170548 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-credential-socket\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.170649 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.170582 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-workload-socket\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.170649 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.170610 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-workload-certs\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.170817 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.170689 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-data\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.170957 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.170936 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istiod-ca-cert\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.172350 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.172330 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-envoy\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.172597 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.172580 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-podinfo\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.178910 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.178886 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79m9l\" (UniqueName: \"kubernetes.io/projected/a10e798a-985e-4ed6-acbe-ec17e9efe53f-kube-api-access-79m9l\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.182098 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.182078 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-token\") pod \"e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.218299 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.218272 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:35.373120 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:35.373084 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g"] Jun 11 19:14:35.376305 ip-10-0-138-156 kubenswrapper[2561]: W0611 19:14:35.376276 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda10e798a_985e_4ed6_acbe_ec17e9efe53f.slice/crio-ed55f3a12ec946d6fcfe2f1d371777b64c2f9ce4c5a7eba9d79c7360a332168f WatchSource:0}: Error finding container ed55f3a12ec946d6fcfe2f1d371777b64c2f9ce4c5a7eba9d79c7360a332168f: Status 404 returned error can't find the container with id ed55f3a12ec946d6fcfe2f1d371777b64c2f9ce4c5a7eba9d79c7360a332168f Jun 11 19:14:36.162318 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:36.162283 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" event={"ID":"a10e798a-985e-4ed6-acbe-ec17e9efe53f","Type":"ContainerStarted","Data":"ed55f3a12ec946d6fcfe2f1d371777b64c2f9ce4c5a7eba9d79c7360a332168f"} Jun 11 19:14:38.220535 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:38.220498 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 19:14:38.220797 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:38.220576 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 19:14:38.220797 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:38.220604 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 19:14:39.178135 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:39.178091 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" event={"ID":"a10e798a-985e-4ed6-acbe-ec17e9efe53f","Type":"ContainerStarted","Data":"6a34b11eea453ee141664893164f7c78b201403e5c4a709d48a9b5d5f50f44a9"} Jun 11 19:14:39.208476 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:39.208422 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" podStartSLOduration=2.36649312 podStartE2EDuration="5.208406166s" totalImagesPullingTime="2.841913046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 19:14:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 19:14:39.207351187 +0000 UTC m=+2320.775424661" watchObservedRunningTime="2026-06-11 19:14:39.208406166 +0000 UTC m=+2320.776479645" Jun 11 19:14:39.219017 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:39.218984 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:39.224305 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:39.224283 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:40.182577 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:40.182540 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:40.183648 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:40.183630 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:41.378465 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:41.378426 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g"] Jun 11 19:14:42.192395 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:42.192354 2561 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" secret="" err="secret \"e2e-ait-3cbd9f4b-openshift-default-dockercfg-rqtsd\" not found" Jun 11 19:14:43.196219 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:43.196150 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" podUID="a10e798a-985e-4ed6-acbe-ec17e9efe53f" containerName="istio-proxy" containerID="cri-o://6a34b11eea453ee141664893164f7c78b201403e5c4a709d48a9b5d5f50f44a9" gracePeriod=30 Jun 11 19:14:48.455112 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.455041 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:48.490125 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490094 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79m9l\" (UniqueName: \"kubernetes.io/projected/a10e798a-985e-4ed6-acbe-ec17e9efe53f-kube-api-access-79m9l\") pod \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " Jun 11 19:14:48.490125 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490128 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-token\") pod \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " Jun 11 19:14:48.490366 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490156 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-workload-socket\") pod \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " Jun 11 19:14:48.490366 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490297 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-credential-socket\") pod \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " Jun 11 19:14:48.490366 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490343 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-data\") pod \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " Jun 11 19:14:48.490522 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490404 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-envoy\") pod \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " Jun 11 19:14:48.490522 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490440 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-workload-certs\") pod \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " Jun 11 19:14:48.490522 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490463 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a10e798a-985e-4ed6-acbe-ec17e9efe53f" (UID: "a10e798a-985e-4ed6-acbe-ec17e9efe53f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 19:14:48.490522 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490471 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-podinfo\") pod \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " Jun 11 19:14:48.490740 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490539 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istiod-ca-cert\") pod \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\" (UID: \"a10e798a-985e-4ed6-acbe-ec17e9efe53f\") " Jun 11 19:14:48.490740 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490545 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a10e798a-985e-4ed6-acbe-ec17e9efe53f" (UID: "a10e798a-985e-4ed6-acbe-ec17e9efe53f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 19:14:48.490740 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490644 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-data" (OuterVolumeSpecName: "istio-data") pod "a10e798a-985e-4ed6-acbe-ec17e9efe53f" (UID: "a10e798a-985e-4ed6-acbe-ec17e9efe53f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 19:14:48.490908 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490765 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a10e798a-985e-4ed6-acbe-ec17e9efe53f" (UID: "a10e798a-985e-4ed6-acbe-ec17e9efe53f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 19:14:48.490965 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490921 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-workload-certs\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:14:48.490965 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490944 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-workload-socket\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:14:48.490965 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490943 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a10e798a-985e-4ed6-acbe-ec17e9efe53f" (UID: "a10e798a-985e-4ed6-acbe-ec17e9efe53f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 19:14:48.490965 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490961 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-credential-socket\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:14:48.491122 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.490976 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-data\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:14:48.492565 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.492541 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a10e798a-985e-4ed6-acbe-ec17e9efe53f" (UID: "a10e798a-985e-4ed6-acbe-ec17e9efe53f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 19:14:48.492636 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.492549 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a10e798a-985e-4ed6-acbe-ec17e9efe53f-kube-api-access-79m9l" (OuterVolumeSpecName: "kube-api-access-79m9l") pod "a10e798a-985e-4ed6-acbe-ec17e9efe53f" (UID: "a10e798a-985e-4ed6-acbe-ec17e9efe53f"). InnerVolumeSpecName "kube-api-access-79m9l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 19:14:48.492738 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.492720 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a10e798a-985e-4ed6-acbe-ec17e9efe53f" (UID: "a10e798a-985e-4ed6-acbe-ec17e9efe53f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 11 19:14:48.492789 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.492755 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-token" (OuterVolumeSpecName: "istio-token") pod "a10e798a-985e-4ed6-acbe-ec17e9efe53f" (UID: "a10e798a-985e-4ed6-acbe-ec17e9efe53f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 19:14:48.591545 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.591515 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79m9l\" (UniqueName: \"kubernetes.io/projected/a10e798a-985e-4ed6-acbe-ec17e9efe53f-kube-api-access-79m9l\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:14:48.591545 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.591542 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-token\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:14:48.591545 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.591553 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-envoy\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:14:48.591784 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.591560 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istio-podinfo\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:14:48.591784 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:48.591569 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a10e798a-985e-4ed6-acbe-ec17e9efe53f-istiod-ca-cert\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:14:49.232756 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.232655 2561 generic.go:356] "Generic (PLEG): container finished" podID="a10e798a-985e-4ed6-acbe-ec17e9efe53f" containerID="6a34b11eea453ee141664893164f7c78b201403e5c4a709d48a9b5d5f50f44a9" exitCode=0 Jun 11 19:14:49.232756 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.232716 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" Jun 11 19:14:49.232756 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.232731 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" event={"ID":"a10e798a-985e-4ed6-acbe-ec17e9efe53f","Type":"ContainerDied","Data":"6a34b11eea453ee141664893164f7c78b201403e5c4a709d48a9b5d5f50f44a9"} Jun 11 19:14:49.233039 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.232812 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g" event={"ID":"a10e798a-985e-4ed6-acbe-ec17e9efe53f","Type":"ContainerDied","Data":"ed55f3a12ec946d6fcfe2f1d371777b64c2f9ce4c5a7eba9d79c7360a332168f"} Jun 11 19:14:49.233039 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.232835 2561 scope.go:117] "RemoveContainer" containerID="6a34b11eea453ee141664893164f7c78b201403e5c4a709d48a9b5d5f50f44a9" Jun 11 19:14:49.241687 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.241665 2561 scope.go:117] "RemoveContainer" containerID="6a34b11eea453ee141664893164f7c78b201403e5c4a709d48a9b5d5f50f44a9" Jun 11 19:14:49.241973 ip-10-0-138-156 kubenswrapper[2561]: E0611 19:14:49.241949 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a34b11eea453ee141664893164f7c78b201403e5c4a709d48a9b5d5f50f44a9\": container with ID starting with 6a34b11eea453ee141664893164f7c78b201403e5c4a709d48a9b5d5f50f44a9 not found: ID does not exist" containerID="6a34b11eea453ee141664893164f7c78b201403e5c4a709d48a9b5d5f50f44a9" Jun 11 19:14:49.242040 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.241982 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a34b11eea453ee141664893164f7c78b201403e5c4a709d48a9b5d5f50f44a9"} err="failed to get container status \"6a34b11eea453ee141664893164f7c78b201403e5c4a709d48a9b5d5f50f44a9\": rpc error: code = NotFound desc = could not find container \"6a34b11eea453ee141664893164f7c78b201403e5c4a709d48a9b5d5f50f44a9\": container with ID starting with 6a34b11eea453ee141664893164f7c78b201403e5c4a709d48a9b5d5f50f44a9 not found: ID does not exist" Jun 11 19:14:49.295816 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.295783 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g"] Jun 11 19:14:49.331577 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.331542 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3cbd9f4b-openshift-default-7f588464c6-vrj5g"] Jun 11 19:14:49.406133 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.406088 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8"] Jun 11 19:14:49.406498 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.406482 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a10e798a-985e-4ed6-acbe-ec17e9efe53f" containerName="istio-proxy" Jun 11 19:14:49.406546 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.406499 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a10e798a-985e-4ed6-acbe-ec17e9efe53f" containerName="istio-proxy" Jun 11 19:14:49.406587 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.406569 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a10e798a-985e-4ed6-acbe-ec17e9efe53f" containerName="istio-proxy" Jun 11 19:14:49.410747 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.410729 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" Jun 11 19:14:49.434690 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.434646 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8"] Jun 11 19:14:49.499101 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.499007 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/901a261a-7921-4f12-acd0-92b23bb2a4da-tmp\") pod \"klusterlet-addon-workmgr-fddfd7768-74sp8\" (UID: \"901a261a-7921-4f12-acd0-92b23bb2a4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" Jun 11 19:14:49.499101 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.499092 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/901a261a-7921-4f12-acd0-92b23bb2a4da-klusterlet-config\") pod \"klusterlet-addon-workmgr-fddfd7768-74sp8\" (UID: \"901a261a-7921-4f12-acd0-92b23bb2a4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" Jun 11 19:14:49.499511 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.499137 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn7sc\" (UniqueName: \"kubernetes.io/projected/901a261a-7921-4f12-acd0-92b23bb2a4da-kube-api-access-wn7sc\") pod \"klusterlet-addon-workmgr-fddfd7768-74sp8\" (UID: \"901a261a-7921-4f12-acd0-92b23bb2a4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" Jun 11 19:14:49.596243 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.596212 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp"] Jun 11 19:14:49.599960 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.599933 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.600257 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.600233 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn7sc\" (UniqueName: \"kubernetes.io/projected/901a261a-7921-4f12-acd0-92b23bb2a4da-kube-api-access-wn7sc\") pod \"klusterlet-addon-workmgr-fddfd7768-74sp8\" (UID: \"901a261a-7921-4f12-acd0-92b23bb2a4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" Jun 11 19:14:49.600374 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.600356 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/901a261a-7921-4f12-acd0-92b23bb2a4da-tmp\") pod \"klusterlet-addon-workmgr-fddfd7768-74sp8\" (UID: \"901a261a-7921-4f12-acd0-92b23bb2a4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" Jun 11 19:14:49.600463 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.600447 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/901a261a-7921-4f12-acd0-92b23bb2a4da-klusterlet-config\") pod \"klusterlet-addon-workmgr-fddfd7768-74sp8\" (UID: \"901a261a-7921-4f12-acd0-92b23bb2a4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" Jun 11 19:14:49.600675 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.600660 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/901a261a-7921-4f12-acd0-92b23bb2a4da-tmp\") pod \"klusterlet-addon-workmgr-fddfd7768-74sp8\" (UID: \"901a261a-7921-4f12-acd0-92b23bb2a4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" Jun 11 19:14:49.602764 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.602739 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/901a261a-7921-4f12-acd0-92b23bb2a4da-klusterlet-config\") pod \"klusterlet-addon-workmgr-fddfd7768-74sp8\" (UID: \"901a261a-7921-4f12-acd0-92b23bb2a4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" Jun 11 19:14:49.605078 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.605062 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-53170be3-openshift-default-dockercfg-fgl2d\"" Jun 11 19:14:49.632848 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.632816 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp"] Jun 11 19:14:49.637933 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.637907 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn7sc\" (UniqueName: \"kubernetes.io/projected/901a261a-7921-4f12-acd0-92b23bb2a4da-kube-api-access-wn7sc\") pod \"klusterlet-addon-workmgr-fddfd7768-74sp8\" (UID: \"901a261a-7921-4f12-acd0-92b23bb2a4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" Jun 11 19:14:49.701202 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.701132 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-workload-socket\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.701202 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.701204 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14e4c25-b654-42ff-8432-14dc63599809-istio-podinfo\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.701442 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.701245 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14e4c25-b654-42ff-8432-14dc63599809-istiod-ca-cert\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.701442 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.701272 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-credential-socket\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.701442 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.701317 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-istio-data\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.701442 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.701361 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14e4c25-b654-42ff-8432-14dc63599809-istio-token\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.701442 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.701402 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-istio-envoy\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.701442 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.701420 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-workload-certs\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.701442 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.701436 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djfmj\" (UniqueName: \"kubernetes.io/projected/f14e4c25-b654-42ff-8432-14dc63599809-kube-api-access-djfmj\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.720151 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.720113 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" Jun 11 19:14:49.802002 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.801958 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-istio-envoy\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.802162 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.802013 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-workload-certs\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.802162 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.802043 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djfmj\" (UniqueName: \"kubernetes.io/projected/f14e4c25-b654-42ff-8432-14dc63599809-kube-api-access-djfmj\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.802162 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.802111 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-workload-socket\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.802162 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.802146 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14e4c25-b654-42ff-8432-14dc63599809-istio-podinfo\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.802669 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.802352 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14e4c25-b654-42ff-8432-14dc63599809-istiod-ca-cert\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.802669 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.802424 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-credential-socket\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.802669 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.802458 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-istio-data\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.802669 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.802476 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-workload-certs\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.802669 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.802513 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14e4c25-b654-42ff-8432-14dc63599809-istio-token\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.802669 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.802513 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-workload-socket\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.803066 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.802768 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-istio-data\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.803066 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.802767 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-credential-socket\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.803579 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.803552 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14e4c25-b654-42ff-8432-14dc63599809-istiod-ca-cert\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.804548 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.804523 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-istio-envoy\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.804952 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.804932 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14e4c25-b654-42ff-8432-14dc63599809-istio-podinfo\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.817072 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.817038 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djfmj\" (UniqueName: \"kubernetes.io/projected/f14e4c25-b654-42ff-8432-14dc63599809-kube-api-access-djfmj\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.817291 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.817202 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14e4c25-b654-42ff-8432-14dc63599809-istio-token\") pod \"e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:49.896783 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.896641 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8"] Jun 11 19:14:49.899605 ip-10-0-138-156 kubenswrapper[2561]: W0611 19:14:49.899578 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod901a261a_7921_4f12_acd0_92b23bb2a4da.slice/crio-14b5d4c03212666cf8aba563458816c3c5952df3e10cc83f9346fc3407059bd0 WatchSource:0}: Error finding container 14b5d4c03212666cf8aba563458816c3c5952df3e10cc83f9346fc3407059bd0: Status 404 returned error can't find the container with id 14b5d4c03212666cf8aba563458816c3c5952df3e10cc83f9346fc3407059bd0 Jun 11 19:14:49.918934 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:49.918906 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:50.095455 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:50.095426 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp"] Jun 11 19:14:50.098139 ip-10-0-138-156 kubenswrapper[2561]: W0611 19:14:50.098109 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf14e4c25_b654_42ff_8432_14dc63599809.slice/crio-9e2379e9f557278dd4a4d7670c9347a26904f13407bbb1ffbe8e8952bd3cf8bd WatchSource:0}: Error finding container 9e2379e9f557278dd4a4d7670c9347a26904f13407bbb1ffbe8e8952bd3cf8bd: Status 404 returned error can't find the container with id 9e2379e9f557278dd4a4d7670c9347a26904f13407bbb1ffbe8e8952bd3cf8bd Jun 11 19:14:50.101204 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:50.101155 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 19:14:50.101297 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:50.101246 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 19:14:50.101297 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:50.101281 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 19:14:50.239895 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:50.239858 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" event={"ID":"f14e4c25-b654-42ff-8432-14dc63599809","Type":"ContainerStarted","Data":"08bbade265eacd2aa49f10eb2531dee0f22ac61bf4153d2999870eb8632aa03a"} Jun 11 19:14:50.240099 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:50.239921 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" event={"ID":"f14e4c25-b654-42ff-8432-14dc63599809","Type":"ContainerStarted","Data":"9e2379e9f557278dd4a4d7670c9347a26904f13407bbb1ffbe8e8952bd3cf8bd"} Jun 11 19:14:50.241138 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:50.241112 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" event={"ID":"901a261a-7921-4f12-acd0-92b23bb2a4da","Type":"ContainerStarted","Data":"14b5d4c03212666cf8aba563458816c3c5952df3e10cc83f9346fc3407059bd0"} Jun 11 19:14:50.285100 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:50.285043 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" podStartSLOduration=1.285025218 podStartE2EDuration="1.285025218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 19:14:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 19:14:50.280917695 +0000 UTC m=+2331.848991174" watchObservedRunningTime="2026-06-11 19:14:50.285025218 +0000 UTC m=+2331.853098700" Jun 11 19:14:50.919955 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:50.919916 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a10e798a-985e-4ed6-acbe-ec17e9efe53f" path="/var/lib/kubelet/pods/a10e798a-985e-4ed6-acbe-ec17e9efe53f/volumes" Jun 11 19:14:50.920518 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:50.920247 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:51.926086 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:51.926055 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:52.253894 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:52.253794 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:52.255035 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:52.255011 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:14:53.261455 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:53.261352 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" event={"ID":"901a261a-7921-4f12-acd0-92b23bb2a4da","Type":"ContainerStarted","Data":"822224eaf4c66158c0c4d21c924703980076b1a722bee9a24cfa64bd4fdfaca6"} Jun 11 19:14:53.261899 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:53.261601 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" Jun 11 19:14:53.263244 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:53.263222 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" Jun 11 19:14:53.302319 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:53.302264 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-fddfd7768-74sp8" podStartSLOduration=1.195392169 podStartE2EDuration="4.302247299s" totalImagesPullingTime="3.10685513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 19:14:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 19:14:53.30125199 +0000 UTC m=+2334.869325471" watchObservedRunningTime="2026-06-11 19:14:53.302247299 +0000 UTC m=+2334.870320779" Jun 11 19:14:53.379500 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:53.379469 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j"] Jun 11 19:14:53.379701 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:53.379677 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" podUID="828f0479-53ba-4372-a7b6-108bc6b8355b" containerName="acm-agent" containerID="cri-o://d40db652c5ae187c8b769d6421cde4074173a76ff00c8a75e55d7bb1d5d99118" gracePeriod=30 Jun 11 19:14:57.039886 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:57.039850 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp"] Jun 11 19:14:57.040338 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:57.040016 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" podUID="f14e4c25-b654-42ff-8432-14dc63599809" containerName="istio-proxy" containerID="cri-o://08bbade265eacd2aa49f10eb2531dee0f22ac61bf4153d2999870eb8632aa03a" gracePeriod=30 Jun 11 19:14:58.579750 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:14:58.579706 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" podUID="828f0479-53ba-4372-a7b6-108bc6b8355b" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.24:8000/readyz\": dial tcp 10.132.0.24:8000: connect: connection refused" Jun 11 19:15:00.187664 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:00.187624 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29686755-h994f"] Jun 11 19:15:00.192473 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:00.192449 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29686755-h994f" Jun 11 19:15:00.197622 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:00.197599 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-s5864\"" Jun 11 19:15:00.204667 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:00.204638 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29686755-h994f"] Jun 11 19:15:00.304373 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:00.304339 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8c5x\" (UniqueName: \"kubernetes.io/projected/1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e-kube-api-access-t8c5x\") pod \"maas-api-key-cleanup-29686755-h994f\" (UID: \"1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e\") " pod="opendatahub/maas-api-key-cleanup-29686755-h994f" Jun 11 19:15:00.405180 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:00.405141 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8c5x\" (UniqueName: \"kubernetes.io/projected/1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e-kube-api-access-t8c5x\") pod \"maas-api-key-cleanup-29686755-h994f\" (UID: \"1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e\") " pod="opendatahub/maas-api-key-cleanup-29686755-h994f" Jun 11 19:15:00.423488 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:00.423455 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8c5x\" (UniqueName: \"kubernetes.io/projected/1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e-kube-api-access-t8c5x\") pod \"maas-api-key-cleanup-29686755-h994f\" (UID: \"1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e\") " pod="opendatahub/maas-api-key-cleanup-29686755-h994f" Jun 11 19:15:00.503894 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:00.503808 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29686755-h994f" Jun 11 19:15:00.674675 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:00.674632 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29686755-h994f"] Jun 11 19:15:00.680198 ip-10-0-138-156 kubenswrapper[2561]: W0611 19:15:00.680157 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bfcf27c_2a0e_4e93_a7ba_f9a326a38d9e.slice/crio-819374f11407a38109f2d9813bbf0eb6d5e0604ee20e16390d725df41fe949ce WatchSource:0}: Error finding container 819374f11407a38109f2d9813bbf0eb6d5e0604ee20e16390d725df41fe949ce: Status 404 returned error can't find the container with id 819374f11407a38109f2d9813bbf0eb6d5e0604ee20e16390d725df41fe949ce Jun 11 19:15:01.304661 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:01.304627 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29686755-h994f" event={"ID":"1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e","Type":"ContainerStarted","Data":"819374f11407a38109f2d9813bbf0eb6d5e0604ee20e16390d725df41fe949ce"} Jun 11 19:15:02.313996 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:02.313903 2561 generic.go:356] "Generic (PLEG): container finished" podID="f14e4c25-b654-42ff-8432-14dc63599809" containerID="08bbade265eacd2aa49f10eb2531dee0f22ac61bf4153d2999870eb8632aa03a" exitCode=0 Jun 11 19:15:02.314421 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:02.313985 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" event={"ID":"f14e4c25-b654-42ff-8432-14dc63599809","Type":"ContainerDied","Data":"08bbade265eacd2aa49f10eb2531dee0f22ac61bf4153d2999870eb8632aa03a"} Jun 11 19:15:02.909454 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:02.909426 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:15:03.028367 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028276 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14e4c25-b654-42ff-8432-14dc63599809-istiod-ca-cert\") pod \"f14e4c25-b654-42ff-8432-14dc63599809\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " Jun 11 19:15:03.028367 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028323 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-istio-data\") pod \"f14e4c25-b654-42ff-8432-14dc63599809\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " Jun 11 19:15:03.028367 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028344 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djfmj\" (UniqueName: \"kubernetes.io/projected/f14e4c25-b654-42ff-8432-14dc63599809-kube-api-access-djfmj\") pod \"f14e4c25-b654-42ff-8432-14dc63599809\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " Jun 11 19:15:03.028620 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028371 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-workload-socket\") pod \"f14e4c25-b654-42ff-8432-14dc63599809\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " Jun 11 19:15:03.028620 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028386 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-workload-certs\") pod \"f14e4c25-b654-42ff-8432-14dc63599809\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " Jun 11 19:15:03.028620 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028435 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-credential-socket\") pod \"f14e4c25-b654-42ff-8432-14dc63599809\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " Jun 11 19:15:03.028620 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028506 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14e4c25-b654-42ff-8432-14dc63599809-istio-podinfo\") pod \"f14e4c25-b654-42ff-8432-14dc63599809\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " Jun 11 19:15:03.028620 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028537 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14e4c25-b654-42ff-8432-14dc63599809-istio-token\") pod \"f14e4c25-b654-42ff-8432-14dc63599809\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " Jun 11 19:15:03.028620 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028570 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-istio-envoy\") pod \"f14e4c25-b654-42ff-8432-14dc63599809\" (UID: \"f14e4c25-b654-42ff-8432-14dc63599809\") " Jun 11 19:15:03.028906 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028728 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-istio-data" (OuterVolumeSpecName: "istio-data") pod "f14e4c25-b654-42ff-8432-14dc63599809" (UID: "f14e4c25-b654-42ff-8432-14dc63599809"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 19:15:03.028906 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028744 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f14e4c25-b654-42ff-8432-14dc63599809" (UID: "f14e4c25-b654-42ff-8432-14dc63599809"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 19:15:03.028906 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028819 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f14e4c25-b654-42ff-8432-14dc63599809" (UID: "f14e4c25-b654-42ff-8432-14dc63599809"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 19:15:03.028906 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028832 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f14e4c25-b654-42ff-8432-14dc63599809" (UID: "f14e4c25-b654-42ff-8432-14dc63599809"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 19:15:03.029132 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028912 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-istio-data\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:15:03.029132 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028919 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f14e4c25-b654-42ff-8432-14dc63599809-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f14e4c25-b654-42ff-8432-14dc63599809" (UID: "f14e4c25-b654-42ff-8432-14dc63599809"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 19:15:03.029132 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028931 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-workload-socket\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:15:03.029132 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028941 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-workload-certs\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:15:03.029132 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.028954 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-credential-socket\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:15:03.030746 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.030724 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f14e4c25-b654-42ff-8432-14dc63599809-kube-api-access-djfmj" (OuterVolumeSpecName: "kube-api-access-djfmj") pod "f14e4c25-b654-42ff-8432-14dc63599809" (UID: "f14e4c25-b654-42ff-8432-14dc63599809"). InnerVolumeSpecName "kube-api-access-djfmj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 19:15:03.030857 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.030772 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f14e4c25-b654-42ff-8432-14dc63599809" (UID: "f14e4c25-b654-42ff-8432-14dc63599809"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 19:15:03.031161 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.031141 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f14e4c25-b654-42ff-8432-14dc63599809-istio-token" (OuterVolumeSpecName: "istio-token") pod "f14e4c25-b654-42ff-8432-14dc63599809" (UID: "f14e4c25-b654-42ff-8432-14dc63599809"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 19:15:03.031257 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.031142 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f14e4c25-b654-42ff-8432-14dc63599809-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f14e4c25-b654-42ff-8432-14dc63599809" (UID: "f14e4c25-b654-42ff-8432-14dc63599809"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 11 19:15:03.130333 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.130298 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14e4c25-b654-42ff-8432-14dc63599809-istio-token\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:15:03.130333 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.130326 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14e4c25-b654-42ff-8432-14dc63599809-istio-envoy\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:15:03.130333 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.130337 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14e4c25-b654-42ff-8432-14dc63599809-istiod-ca-cert\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:15:03.130547 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.130345 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djfmj\" (UniqueName: \"kubernetes.io/projected/f14e4c25-b654-42ff-8432-14dc63599809-kube-api-access-djfmj\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:15:03.130547 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.130355 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14e4c25-b654-42ff-8432-14dc63599809-istio-podinfo\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:15:03.319914 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.319889 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" Jun 11 19:15:03.320463 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.319936 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp" event={"ID":"f14e4c25-b654-42ff-8432-14dc63599809","Type":"ContainerDied","Data":"9e2379e9f557278dd4a4d7670c9347a26904f13407bbb1ffbe8e8952bd3cf8bd"} Jun 11 19:15:03.320463 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.320002 2561 scope.go:117] "RemoveContainer" containerID="08bbade265eacd2aa49f10eb2531dee0f22ac61bf4153d2999870eb8632aa03a" Jun 11 19:15:03.321605 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.321581 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29686755-h994f" event={"ID":"1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e","Type":"ContainerStarted","Data":"6baeb3ae3205a14da0fda7aa685539cb6d7f20d956bb4bf1f61e93bdbdf14969"} Jun 11 19:15:03.374342 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.374307 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp"] Jun 11 19:15:03.386193 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.385030 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-53170be3-openshift-default-85f6d845bb-d8lrp"] Jun 11 19:15:03.419499 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:03.419450 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29686755-h994f" podStartSLOduration=1.277633209 podStartE2EDuration="3.419434551s" totalImagesPullingTime="2.141801342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 19:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 19:15:03.41862121 +0000 UTC m=+2344.986694711" watchObservedRunningTime="2026-06-11 19:15:03.419434551 +0000 UTC m=+2344.987508031" Jun 11 19:15:04.328406 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:04.328370 2561 generic.go:356] "Generic (PLEG): container finished" podID="1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e" containerID="6baeb3ae3205a14da0fda7aa685539cb6d7f20d956bb4bf1f61e93bdbdf14969" exitCode=0 Jun 11 19:15:04.328851 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:04.328454 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29686755-h994f" event={"ID":"1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e","Type":"ContainerDied","Data":"6baeb3ae3205a14da0fda7aa685539cb6d7f20d956bb4bf1f61e93bdbdf14969"} Jun 11 19:15:04.919263 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:04.919229 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f14e4c25-b654-42ff-8432-14dc63599809" path="/var/lib/kubelet/pods/f14e4c25-b654-42ff-8432-14dc63599809/volumes" Jun 11 19:15:05.480423 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:05.480397 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29686755-h994f" Jun 11 19:15:05.553423 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:05.553389 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8c5x\" (UniqueName: \"kubernetes.io/projected/1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e-kube-api-access-t8c5x\") pod \"1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e\" (UID: \"1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e\") " Jun 11 19:15:05.555687 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:05.555655 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e-kube-api-access-t8c5x" (OuterVolumeSpecName: "kube-api-access-t8c5x") pod "1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e" (UID: "1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e"). InnerVolumeSpecName "kube-api-access-t8c5x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 19:15:05.654412 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:05.654324 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8c5x\" (UniqueName: \"kubernetes.io/projected/1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e-kube-api-access-t8c5x\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:15:06.340280 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:06.340245 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29686755-h994f" Jun 11 19:15:06.340437 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:06.340283 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29686755-h994f" event={"ID":"1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e","Type":"ContainerDied","Data":"819374f11407a38109f2d9813bbf0eb6d5e0604ee20e16390d725df41fe949ce"} Jun 11 19:15:06.340437 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:06.340327 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="819374f11407a38109f2d9813bbf0eb6d5e0604ee20e16390d725df41fe949ce" Jun 11 19:15:08.579425 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:08.579376 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" podUID="828f0479-53ba-4372-a7b6-108bc6b8355b" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.24:8000/readyz\": dial tcp 10.132.0.24:8000: connect: connection refused" Jun 11 19:15:17.681969 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:17.681935 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-zgv4d_31721fa7-6100-41c0-a64c-62498ab0e81a/manager/0.log" Jun 11 19:15:18.190724 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:18.190696 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29686755-h994f_1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e/cleanup/0.log" Jun 11 19:15:18.452254 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:18.452148 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-scn5g_2744b8dd-170c-433e-b137-c5bdbc2ab252/server/0.log" Jun 11 19:15:18.579811 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:18.579768 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" podUID="828f0479-53ba-4372-a7b6-108bc6b8355b" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.24:8000/readyz\": dial tcp 10.132.0.24:8000: connect: connection refused" Jun 11 19:15:18.579990 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:18.579891 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 19:15:19.012276 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:19.012243 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54d5cbc44f-vr2sr_6a2590cc-b669-4c75-aa43-caf07a2615d1/copy-manifests/0.log" Jun 11 19:15:19.020804 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:19.020779 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54d5cbc44f-vr2sr_6a2590cc-b669-4c75-aa43-caf07a2615d1/manager/0.log" Jun 11 19:15:19.149141 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:19.149110 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-z9pbp_dd4f0814-5848-4adf-b334-e9cf64d511cb/postgres/0.log" Jun 11 19:15:22.091481 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:22.091449 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-wlhf4_241dca3f-d650-4f38-94c0-5b0b94a17619/discovery/0.log" Jun 11 19:15:22.219107 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:22.219081 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8d878499b-92tdm_970e8ea0-a08b-46ea-badb-f9d914f344a1/kube-auth-proxy/0.log" Jun 11 19:15:22.729683 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:22.729648 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-jghh4_e6180710-f1b3-4c0a-8d79-4ca0c7dd7245/payload-processing/0.log" Jun 11 19:15:22.863733 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:22.863703 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-85b5fb789b-jt8gg_c1651923-19d8-41b4-8f23-21afb8ac43a7/router/0.log" Jun 11 19:15:23.394124 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.394094 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-58sj7_0caea8f3-61a6-480b-8885-4ec8561b6f5d/main/0.log" Jun 11 19:15:23.403368 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.403341 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-58sj7_0caea8f3-61a6-480b-8885-4ec8561b6f5d/storage-initializer/0.log" Jun 11 19:15:23.432536 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.432499 2561 generic.go:356] "Generic (PLEG): container finished" podID="828f0479-53ba-4372-a7b6-108bc6b8355b" containerID="d40db652c5ae187c8b769d6421cde4074173a76ff00c8a75e55d7bb1d5d99118" exitCode=137 Jun 11 19:15:23.432705 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.432575 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" event={"ID":"828f0479-53ba-4372-a7b6-108bc6b8355b","Type":"ContainerDied","Data":"d40db652c5ae187c8b769d6421cde4074173a76ff00c8a75e55d7bb1d5d99118"} Jun 11 19:15:23.535973 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.535948 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 19:15:23.616932 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.616895 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/828f0479-53ba-4372-a7b6-108bc6b8355b-tmp\") pod \"828f0479-53ba-4372-a7b6-108bc6b8355b\" (UID: \"828f0479-53ba-4372-a7b6-108bc6b8355b\") " Jun 11 19:15:23.617107 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.616989 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ws9n\" (UniqueName: \"kubernetes.io/projected/828f0479-53ba-4372-a7b6-108bc6b8355b-kube-api-access-5ws9n\") pod \"828f0479-53ba-4372-a7b6-108bc6b8355b\" (UID: \"828f0479-53ba-4372-a7b6-108bc6b8355b\") " Jun 11 19:15:23.617107 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.617033 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/828f0479-53ba-4372-a7b6-108bc6b8355b-klusterlet-config\") pod \"828f0479-53ba-4372-a7b6-108bc6b8355b\" (UID: \"828f0479-53ba-4372-a7b6-108bc6b8355b\") " Jun 11 19:15:23.617250 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.617199 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/828f0479-53ba-4372-a7b6-108bc6b8355b-tmp" (OuterVolumeSpecName: "tmp") pod "828f0479-53ba-4372-a7b6-108bc6b8355b" (UID: "828f0479-53ba-4372-a7b6-108bc6b8355b"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 19:15:23.617372 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.617356 2561 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/828f0479-53ba-4372-a7b6-108bc6b8355b-tmp\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:15:23.619017 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.618995 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/828f0479-53ba-4372-a7b6-108bc6b8355b-kube-api-access-5ws9n" (OuterVolumeSpecName: "kube-api-access-5ws9n") pod "828f0479-53ba-4372-a7b6-108bc6b8355b" (UID: "828f0479-53ba-4372-a7b6-108bc6b8355b"). InnerVolumeSpecName "kube-api-access-5ws9n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 19:15:23.619188 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.619156 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/828f0479-53ba-4372-a7b6-108bc6b8355b-klusterlet-config" (OuterVolumeSpecName: "klusterlet-config") pod "828f0479-53ba-4372-a7b6-108bc6b8355b" (UID: "828f0479-53ba-4372-a7b6-108bc6b8355b"). InnerVolumeSpecName "klusterlet-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 19:15:23.717923 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.717831 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ws9n\" (UniqueName: \"kubernetes.io/projected/828f0479-53ba-4372-a7b6-108bc6b8355b-kube-api-access-5ws9n\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:15:23.717923 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.717865 2561 reconciler_common.go:299] "Volume detached for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/828f0479-53ba-4372-a7b6-108bc6b8355b-klusterlet-config\") on node \"ip-10-0-138-156.ec2.internal\" DevicePath \"\"" Jun 11 19:15:23.814866 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.814833 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-dln9j_00ab0fc2-d0ce-48c7-a450-955b22b0e9b5/storage-initializer/0.log" Jun 11 19:15:23.825936 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.825906 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-dln9j_00ab0fc2-d0ce-48c7-a450-955b22b0e9b5/main/0.log" Jun 11 19:15:23.957457 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.957420 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-r9hl5_86aa963a-1190-477c-ac13-3fec1f93da05/storage-initializer/0.log" Jun 11 19:15:23.968477 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:23.968406 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-r9hl5_86aa963a-1190-477c-ac13-3fec1f93da05/main/0.log" Jun 11 19:15:24.438662 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:24.438632 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" Jun 11 19:15:24.438662 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:24.438641 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j" event={"ID":"828f0479-53ba-4372-a7b6-108bc6b8355b","Type":"ContainerDied","Data":"b4cc830e9d5e030426fe619c4d8e1f14fa9cb995a9757174304e43dfd320c32c"} Jun 11 19:15:24.439112 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:24.438702 2561 scope.go:117] "RemoveContainer" containerID="d40db652c5ae187c8b769d6421cde4074173a76ff00c8a75e55d7bb1d5d99118" Jun 11 19:15:24.474898 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:24.474868 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j"] Jun 11 19:15:24.478794 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:24.478772 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7df8d85848-xs64j"] Jun 11 19:15:24.918996 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:24.918965 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="828f0479-53ba-4372-a7b6-108bc6b8355b" path="/var/lib/kubelet/pods/828f0479-53ba-4372-a7b6-108bc6b8355b/volumes" Jun 11 19:15:28.938568 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:28.938532 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fwsb7/must-gather-dfd6b"] Jun 11 19:15:28.938966 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:28.938913 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e" containerName="cleanup" Jun 11 19:15:28.938966 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:28.938925 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e" containerName="cleanup" Jun 11 19:15:28.938966 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:28.938939 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="828f0479-53ba-4372-a7b6-108bc6b8355b" containerName="acm-agent" Jun 11 19:15:28.938966 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:28.938945 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="828f0479-53ba-4372-a7b6-108bc6b8355b" containerName="acm-agent" Jun 11 19:15:28.938966 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:28.938954 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f14e4c25-b654-42ff-8432-14dc63599809" containerName="istio-proxy" Jun 11 19:15:28.938966 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:28.938960 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f14e4c25-b654-42ff-8432-14dc63599809" containerName="istio-proxy" Jun 11 19:15:28.939158 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:28.939019 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e" containerName="cleanup" Jun 11 19:15:28.939158 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:28.939029 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f14e4c25-b654-42ff-8432-14dc63599809" containerName="istio-proxy" Jun 11 19:15:28.939158 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:28.939037 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="828f0479-53ba-4372-a7b6-108bc6b8355b" containerName="acm-agent" Jun 11 19:15:28.942726 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:28.942708 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fwsb7/must-gather-dfd6b" Jun 11 19:15:28.946015 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:28.945987 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fwsb7\"/\"kube-root-ca.crt\"" Jun 11 19:15:28.946467 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:28.946451 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fwsb7\"/\"openshift-service-ca.crt\"" Jun 11 19:15:28.946589 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:28.946487 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fwsb7\"/\"default-dockercfg-74dfg\"" Jun 11 19:15:28.952964 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:28.952937 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fwsb7/must-gather-dfd6b"] Jun 11 19:15:29.065847 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:29.065810 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/78676f60-ee63-4086-b408-d0619e0a13e9-must-gather-output\") pod \"must-gather-dfd6b\" (UID: \"78676f60-ee63-4086-b408-d0619e0a13e9\") " pod="openshift-must-gather-fwsb7/must-gather-dfd6b" Jun 11 19:15:29.066021 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:29.065921 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvnn4\" (UniqueName: \"kubernetes.io/projected/78676f60-ee63-4086-b408-d0619e0a13e9-kube-api-access-qvnn4\") pod \"must-gather-dfd6b\" (UID: \"78676f60-ee63-4086-b408-d0619e0a13e9\") " pod="openshift-must-gather-fwsb7/must-gather-dfd6b" Jun 11 19:15:29.167248 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:29.167213 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvnn4\" (UniqueName: \"kubernetes.io/projected/78676f60-ee63-4086-b408-d0619e0a13e9-kube-api-access-qvnn4\") pod \"must-gather-dfd6b\" (UID: \"78676f60-ee63-4086-b408-d0619e0a13e9\") " pod="openshift-must-gather-fwsb7/must-gather-dfd6b" Jun 11 19:15:29.167423 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:29.167277 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/78676f60-ee63-4086-b408-d0619e0a13e9-must-gather-output\") pod \"must-gather-dfd6b\" (UID: \"78676f60-ee63-4086-b408-d0619e0a13e9\") " pod="openshift-must-gather-fwsb7/must-gather-dfd6b" Jun 11 19:15:29.167612 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:29.167595 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/78676f60-ee63-4086-b408-d0619e0a13e9-must-gather-output\") pod \"must-gather-dfd6b\" (UID: \"78676f60-ee63-4086-b408-d0619e0a13e9\") " pod="openshift-must-gather-fwsb7/must-gather-dfd6b" Jun 11 19:15:29.181737 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:29.181711 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvnn4\" (UniqueName: \"kubernetes.io/projected/78676f60-ee63-4086-b408-d0619e0a13e9-kube-api-access-qvnn4\") pod \"must-gather-dfd6b\" (UID: \"78676f60-ee63-4086-b408-d0619e0a13e9\") " pod="openshift-must-gather-fwsb7/must-gather-dfd6b" Jun 11 19:15:29.270101 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:29.270009 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fwsb7/must-gather-dfd6b" Jun 11 19:15:29.434337 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:29.434302 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fwsb7/must-gather-dfd6b"] Jun 11 19:15:29.437590 ip-10-0-138-156 kubenswrapper[2561]: W0611 19:15:29.437561 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78676f60_ee63_4086_b408_d0619e0a13e9.slice/crio-ea60b69295946aa670c7159ecf2a7f8e1b5721b041eeb94a788a7d7d34763340 WatchSource:0}: Error finding container ea60b69295946aa670c7159ecf2a7f8e1b5721b041eeb94a788a7d7d34763340: Status 404 returned error can't find the container with id ea60b69295946aa670c7159ecf2a7f8e1b5721b041eeb94a788a7d7d34763340 Jun 11 19:15:29.466830 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:29.466796 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fwsb7/must-gather-dfd6b" event={"ID":"78676f60-ee63-4086-b408-d0619e0a13e9","Type":"ContainerStarted","Data":"ea60b69295946aa670c7159ecf2a7f8e1b5721b041eeb94a788a7d7d34763340"} Jun 11 19:15:30.475864 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:30.475767 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fwsb7/must-gather-dfd6b" event={"ID":"78676f60-ee63-4086-b408-d0619e0a13e9","Type":"ContainerStarted","Data":"19862707767c3477472eb36a28bfe1acccddbf9fff46ef14a852b0583948c74b"} Jun 11 19:15:30.475864 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:30.475817 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fwsb7/must-gather-dfd6b" event={"ID":"78676f60-ee63-4086-b408-d0619e0a13e9","Type":"ContainerStarted","Data":"d6973d2209aab98d6c1ce39f1476e749db8e174ee064e2eb3652abd5d1718669"} Jun 11 19:15:30.519141 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:30.519075 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fwsb7/must-gather-dfd6b" podStartSLOduration=1.765215382 podStartE2EDuration="2.519051526s" totalImagesPullingTime="753.836144ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 19:15:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 19:15:30.517866473 +0000 UTC m=+2372.085939970" watchObservedRunningTime="2026-06-11 19:15:30.519051526 +0000 UTC m=+2372.087125007" Jun 11 19:15:32.595634 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:32.595598 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-58lbd_fb731d8a-734b-48f1-87a2-303368226673/global-pull-secret-syncer/0.log" Jun 11 19:15:33.005308 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:33.005232 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-k8zpf_21a1280a-bea2-4fe2-8795-a55d4dc7c65b/konnectivity-agent/0.log" Jun 11 19:15:33.051656 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:33.051625 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-156.ec2.internal_536524fd725acf153e94591109a49596/haproxy/0.log" Jun 11 19:15:40.155682 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:40.155639 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-f9d7df769-85bbt_08e48b3b-9d88-4715-aa6a-008e8cdebb3e/cluster-monitoring-operator/0.log" Jun 11 19:15:40.510044 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:40.509696 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5zhj9_aad367a5-3066-4955-9163-578131a93d8e/node-exporter/0.log" Jun 11 19:15:40.555380 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:40.555347 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5zhj9_aad367a5-3066-4955-9163-578131a93d8e/kube-rbac-proxy/0.log" Jun 11 19:15:40.602891 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:40.602859 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5zhj9_aad367a5-3066-4955-9163-578131a93d8e/init-textfile/0.log" Jun 11 19:15:41.527584 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:41.527553 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5cf667749c-kqcj8_3f9034ca-6ad4-4c91-8103-55ebb839277f/prometheus-operator-admission-webhook/0.log" Jun 11 19:15:43.223133 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:43.223099 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-867b76cb74-szglk_3745889d-fbde-41e9-b083-3a33484ded3a/networking-console-plugin/0.log" Jun 11 19:15:43.872920 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:43.872886 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 19:15:43.884032 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:43.883994 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/2.log" Jun 11 19:15:44.494645 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.494610 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-694c955b9-9xh9f_84d86e1c-4a2f-4568-aaf1-d14b11521f53/download-server/0.log" Jun 11 19:15:44.506451 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.506414 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq"] Jun 11 19:15:44.511222 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.511194 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.522565 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.522536 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq"] Jun 11 19:15:44.540783 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.540750 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e73abd8a-3ae0-4328-8bb3-7cb6f75396e9-proc\") pod \"perf-node-gather-daemonset-dr8mq\" (UID: \"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9\") " pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.541040 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.541022 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e73abd8a-3ae0-4328-8bb3-7cb6f75396e9-sys\") pod \"perf-node-gather-daemonset-dr8mq\" (UID: \"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9\") " pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.541223 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.541206 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmqgm\" (UniqueName: \"kubernetes.io/projected/e73abd8a-3ae0-4328-8bb3-7cb6f75396e9-kube-api-access-dmqgm\") pod \"perf-node-gather-daemonset-dr8mq\" (UID: \"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9\") " pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.541397 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.541379 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e73abd8a-3ae0-4328-8bb3-7cb6f75396e9-podres\") pod \"perf-node-gather-daemonset-dr8mq\" (UID: \"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9\") " pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.541544 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.541529 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e73abd8a-3ae0-4328-8bb3-7cb6f75396e9-lib-modules\") pod \"perf-node-gather-daemonset-dr8mq\" (UID: \"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9\") " pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.642462 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.642425 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e73abd8a-3ae0-4328-8bb3-7cb6f75396e9-podres\") pod \"perf-node-gather-daemonset-dr8mq\" (UID: \"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9\") " pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.642772 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.642732 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e73abd8a-3ae0-4328-8bb3-7cb6f75396e9-podres\") pod \"perf-node-gather-daemonset-dr8mq\" (UID: \"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9\") " pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.642920 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.642898 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e73abd8a-3ae0-4328-8bb3-7cb6f75396e9-lib-modules\") pod \"perf-node-gather-daemonset-dr8mq\" (UID: \"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9\") " pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.643004 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.642776 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e73abd8a-3ae0-4328-8bb3-7cb6f75396e9-lib-modules\") pod \"perf-node-gather-daemonset-dr8mq\" (UID: \"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9\") " pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.643148 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.643130 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e73abd8a-3ae0-4328-8bb3-7cb6f75396e9-proc\") pod \"perf-node-gather-daemonset-dr8mq\" (UID: \"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9\") " pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.643246 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.643188 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e73abd8a-3ae0-4328-8bb3-7cb6f75396e9-sys\") pod \"perf-node-gather-daemonset-dr8mq\" (UID: \"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9\") " pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.643306 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.643251 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmqgm\" (UniqueName: \"kubernetes.io/projected/e73abd8a-3ae0-4328-8bb3-7cb6f75396e9-kube-api-access-dmqgm\") pod \"perf-node-gather-daemonset-dr8mq\" (UID: \"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9\") " pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.643675 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.643648 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e73abd8a-3ae0-4328-8bb3-7cb6f75396e9-proc\") pod \"perf-node-gather-daemonset-dr8mq\" (UID: \"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9\") " pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.643754 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.643714 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e73abd8a-3ae0-4328-8bb3-7cb6f75396e9-sys\") pod \"perf-node-gather-daemonset-dr8mq\" (UID: \"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9\") " pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.651253 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.651228 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmqgm\" (UniqueName: \"kubernetes.io/projected/e73abd8a-3ae0-4328-8bb3-7cb6f75396e9-kube-api-access-dmqgm\") pod \"perf-node-gather-daemonset-dr8mq\" (UID: \"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9\") " pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:44.826282 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:44.826244 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:45.013029 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:45.011219 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq"] Jun 11 19:15:45.014759 ip-10-0-138-156 kubenswrapper[2561]: W0611 19:15:45.014723 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode73abd8a_3ae0_4328_8bb3_7cb6f75396e9.slice/crio-20abdb9c21c99b4abbf86490abf5e9612ac0c40a71250556fd43c1f1b7d3340b WatchSource:0}: Error finding container 20abdb9c21c99b4abbf86490abf5e9612ac0c40a71250556fd43c1f1b7d3340b: Status 404 returned error can't find the container with id 20abdb9c21c99b4abbf86490abf5e9612ac0c40a71250556fd43c1f1b7d3340b Jun 11 19:15:45.171742 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:45.171703 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-bd4ccd799-p9n54_51cdc3b6-a443-43aa-b6c2-def980560f1b/volume-data-source-validator/0.log" Jun 11 19:15:45.585023 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:45.584993 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" event={"ID":"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9","Type":"ContainerStarted","Data":"2717ed6a4608249c40cd536658daddbbdafef668b81cd5ebf2334a9232c4242f"} Jun 11 19:15:45.585453 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:45.585041 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" event={"ID":"e73abd8a-3ae0-4328-8bb3-7cb6f75396e9","Type":"ContainerStarted","Data":"20abdb9c21c99b4abbf86490abf5e9612ac0c40a71250556fd43c1f1b7d3340b"} Jun 11 19:15:45.585453 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:45.585092 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:45.602711 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:45.602661 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" podStartSLOduration=1.602638784 podStartE2EDuration="1.602638784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 19:15:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 19:15:45.601043824 +0000 UTC m=+2387.169117306" watchObservedRunningTime="2026-06-11 19:15:45.602638784 +0000 UTC m=+2387.170712318" Jun 11 19:15:46.411808 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:46.411778 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pxnqh_e42d52fc-e45a-415f-9694-26853b83c2be/dns/0.log" Jun 11 19:15:46.462729 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:46.462696 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pxnqh_e42d52fc-e45a-415f-9694-26853b83c2be/kube-rbac-proxy/0.log" Jun 11 19:15:46.549305 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:46.549235 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w4hqm_3bac4fe8-0f72-4e10-8c74-3df40f3e6a05/dns-node-resolver/0.log" Jun 11 19:15:47.254510 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:47.254480 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6f5b8b6fc8-97jhj_ef47c69c-b12b-4ce7-af1f-84f267eba571/registry/0.log" Jun 11 19:15:47.281004 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:47.280975 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jx59l_ed57ef82-e664-4ffd-b72c-3fc0422eacab/node-ca/0.log" Jun 11 19:15:48.581234 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:48.581203 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-wlhf4_241dca3f-d650-4f38-94c0-5b0b94a17619/discovery/0.log" Jun 11 19:15:48.608632 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:48.608607 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8d878499b-92tdm_970e8ea0-a08b-46ea-badb-f9d914f344a1/kube-auth-proxy/0.log" Jun 11 19:15:48.864072 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:48.863972 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-jghh4_e6180710-f1b3-4c0a-8d79-4ca0c7dd7245/payload-processing/0.log" Jun 11 19:15:48.992012 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:48.991986 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-85b5fb789b-jt8gg_c1651923-19d8-41b4-8f23-21afb8ac43a7/router/0.log" Jun 11 19:15:49.400290 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:49.400244 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kube-system/konnectivity-agent-k8zpf" podUID="21a1280a-bea2-4fe2-8795-a55d4dc7c65b" containerName="konnectivity-agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 19:15:49.400486 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:49.400359 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-k8zpf" Jun 11 19:15:49.401242 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:49.401202 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kube-system/konnectivity-agent-k8zpf" podUID="21a1280a-bea2-4fe2-8795-a55d4dc7c65b" containerName="konnectivity-agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 19:15:49.771279 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:49.771198 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tt7bw_6f2b34d7-67c4-4695-b705-40f852c4f63f/serve-healthcheck-canary/0.log" Jun 11 19:15:50.309186 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:50.309120 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5bbd86d6bd-h7sf5_64cd27df-ed04-40a9-89c5-0df92402e31b/insights-operator/0.log" Jun 11 19:15:50.311023 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:50.310997 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5bbd86d6bd-h7sf5_64cd27df-ed04-40a9-89c5-0df92402e31b/insights-operator/1.log" Jun 11 19:15:50.334540 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:50.334515 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jjtxg_0f504b49-7bd7-4d7e-8de4-bcc282566088/kube-rbac-proxy/0.log" Jun 11 19:15:50.363876 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:50.363851 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jjtxg_0f504b49-7bd7-4d7e-8de4-bcc282566088/exporter/0.log" Jun 11 19:15:50.388418 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:50.388387 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jjtxg_0f504b49-7bd7-4d7e-8de4-bcc282566088/extractor/0.log" Jun 11 19:15:51.603337 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:51.603311 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fwsb7/perf-node-gather-daemonset-dr8mq" Jun 11 19:15:52.994319 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:52.994290 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-zgv4d_31721fa7-6100-41c0-a64c-62498ab0e81a/manager/0.log" Jun 11 19:15:53.219790 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:53.219768 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29686755-h994f_1bfcf27c-2a0e-4e93-a7ba-f9a326a38d9e/cleanup/0.log" Jun 11 19:15:53.329616 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:53.329586 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-scn5g_2744b8dd-170c-433e-b137-c5bdbc2ab252/server/0.log" Jun 11 19:15:53.613689 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:53.613564 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54d5cbc44f-vr2sr_6a2590cc-b669-4c75-aa43-caf07a2615d1/manager/0.log" Jun 11 19:15:53.644145 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:53.644122 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54d5cbc44f-vr2sr_6a2590cc-b669-4c75-aa43-caf07a2615d1/copy-manifests/0.log" Jun 11 19:15:53.674270 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:53.674224 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-z9pbp_dd4f0814-5848-4adf-b334-e9cf64d511cb/postgres/0.log" Jun 11 19:15:57.879793 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:57.879762 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-wlhf4_241dca3f-d650-4f38-94c0-5b0b94a17619/discovery/0.log" Jun 11 19:15:57.889809 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:57.889784 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8d878499b-92tdm_970e8ea0-a08b-46ea-badb-f9d914f344a1/kube-auth-proxy/0.log" Jun 11 19:15:58.087708 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:58.087672 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-jghh4_e6180710-f1b3-4c0a-8d79-4ca0c7dd7245/payload-processing/0.log" Jun 11 19:15:58.103006 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:58.102975 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-85b5fb789b-jt8gg_c1651923-19d8-41b4-8f23-21afb8ac43a7/router/0.log" Jun 11 19:15:59.090162 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:59.090133 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 19:15:59.090162 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:59.090133 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-kpstz_d4de0340-920b-472d-8ff1-0f6b69840a5e/console-operator/1.log" Jun 11 19:15:59.097455 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:59.097434 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 19:15:59.097610 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:59.097503 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhtcz_597fee71-168a-4df0-b3c0-6d65f17a570f/ovn-acl-logging/0.log" Jun 11 19:15:59.364211 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:59.364115 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-wlhf4_241dca3f-d650-4f38-94c0-5b0b94a17619/discovery/0.log" Jun 11 19:15:59.376743 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:59.376719 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8d878499b-92tdm_970e8ea0-a08b-46ea-badb-f9d914f344a1/kube-auth-proxy/0.log" Jun 11 19:15:59.586233 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:59.586196 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-jghh4_e6180710-f1b3-4c0a-8d79-4ca0c7dd7245/payload-processing/0.log" Jun 11 19:15:59.609865 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:15:59.609835 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-85b5fb789b-jt8gg_c1651923-19d8-41b4-8f23-21afb8ac43a7/router/0.log" Jun 11 19:16:01.152125 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:16:01.152095 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d4bcd4867-6pz4b_b3f6a6e3-662f-4b78-b930-f87e6e78c8c6/migrator/0.log" Jun 11 19:16:01.177719 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:16:01.177693 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d4bcd4867-6pz4b_b3f6a6e3-662f-4b78-b930-f87e6e78c8c6/graceful-termination/0.log" Jun 11 19:16:01.563653 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:16:01.563625 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-wlhf4_241dca3f-d650-4f38-94c0-5b0b94a17619/discovery/0.log" Jun 11 19:16:01.581802 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:16:01.581777 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8d878499b-92tdm_970e8ea0-a08b-46ea-badb-f9d914f344a1/kube-auth-proxy/0.log" Jun 11 19:16:01.597118 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:16:01.597089 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7c578b78fc-6kgqh_6df2530a-816a-4737-8815-4c537d8a61c4/kube-storage-version-migrator-operator/1.log" Jun 11 19:16:01.598243 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:16:01.598217 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7c578b78fc-6kgqh_6df2530a-816a-4737-8815-4c537d8a61c4/kube-storage-version-migrator-operator/0.log" Jun 11 19:16:01.752780 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:16:01.752698 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-jghh4_e6180710-f1b3-4c0a-8d79-4ca0c7dd7245/payload-processing/0.log" Jun 11 19:16:01.765416 ip-10-0-138-156 kubenswrapper[2561]: I0611 19:16:01.765393 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-85b5fb789b-jt8gg_c1651923-19d8-41b4-8f23-21afb8ac43a7/router/0.log"